from:"Theodoros Theodoropoulos"


Hello all,

Strange as it might seem, I'm trying to send an email from a webfunction 
to two recipients and it's only delivered to the first one!


I tried the same addresses using mailutils in ipython:

from invenio import mailutils
mailutils.send_email([serverurl],[email1],[email2],subject=test,content=test 
content, debug_level=9)


and I see that it's only send to the FIRST email address! (BTW, both 
emails are valid and working :) but i )


Having said that, when I enable CFG_DEVEL_SITE=1 and run THE SAME command,
as an admin I get an email that shows both addresses, so it's not an 
issue of splitting the email strings etc, but something deeper... (maybe 
python-related?)


Can you verify it too? It seems a bit strange to be a bug and not having 
been reported by anyone else, on the other hand, I cannot find anything 
wrong with my tests... Hmmm...


Best regards,
Theodoros

Re: mailutils.send_mail with multiple recipients sends mail only to the first recipient


On 5/12/2013 7:06 μμ, Theodoros Theodoropoulos wrote:

from invenio import mailutils
mailutils.send_email([serverurl],[email1],[email2],subject=test,content=test 
content, debug_level=9)
and I see that it's only send to the FIRST email address! (BTW, both 
emails are valid and working :) but i )

FYI, the above SAME code in invenio 0.99.x branch works without problems!!

If you can verify my findings, I think I should create a ticket with 
'critical' priority (because mails could be lost if not fixed)

(I also tried using list of strings in the replyto, with the same results)

Cheers,
Theodoros

Re: mailutils.send_mail with multiple recipients sends mail only to the first recipient


Correction:

(I also tried using list of strings in the TOADDR, with the same results)
meaning: mailutils.send_email(serverurl, [email1,email2], 
subject=test, content=test content, debug_level=9)

Re: mailutils.send_mail with multiple recipients sends mail only to the first recipient [FOUND IT]


I found the culprit! It's remove_temporary_emails() !

Although the function works as expected (tried it), it changes 'toaddr' 
from list of strings, to strings.


Here's the code:

if type(toaddr) is str:
toaddr = toaddr.strip().split(',')

up to here toaddr it's definitely list of strings...

toaddr = remove_temporary_emails(toaddr)

Here it's a simple string

Then things start to not work as expected.

If I comment out remove_temporary_emails, everything works as expected.

I'll post a ticket. remove_temporary_emails should return list of strings.


On 6/12/2013 8:39 πμ, Theodoros Theodoropoulos wrote:

Correction:
(I also tried using list of strings in the TOADDR, with the same 
results)
meaning: mailutils.send_email(serverurl, [email1,email2], 
subject=test, content=test content, debug_level=9)

(simple?) search question

2013-12-03 Thread Theodoros Theodoropoulos


Hello everyone,

I have this situation:

Say I have the following marc:

700__ $$alastname1, firstname1$$eauthor
700__ $$alastname2, firstname2$$eeditor

and i want to find all records where author is lastname1, firstname1 
and THIS PERSON is editor


Provided that i have an author index with 100/700 a and e logical 
fields, if I use the search query author:editor and author:lastname1, 
firstname1, the above record will be returned, but it shouldn't.


So, briefly, what I need is a type of sub-search in the same field as 
the first argument... Is there a way (using regular expressions or a 
trick/hack) to make this work and get some proper results?


Thanks in advance,
Theodoros

Re: (simple?) search question

2013-12-03 Thread Theodoros Theodoropoulos


On 3/12/2013 4:23 μμ, Tibor Simko wrote:

On Tue, 03 Dec 2013, th...@physics.auth.gr wrote:

So, briefly, what I need is a type of sub-search in the same field as
the first argument... Is there a way (using regular expressions or a
trick/hack) to make this work and get some proper results?

Right now you'd have to make a second pass: firstly, retrieve results as
you do now (including false positives); secondly, pass through results
calling field instance filtering function that would pick only the
fields having wanted another subfield.

WRT filtering, see:

   http://invenio-software.org/ticket/1550

and the related commits.

So, while this is doable, and the master branch has ingredients ready,
there is some code to write and run after your searches, which may be
too slow in case there is a huge number of results returned in the first
step before the filtering step is called.
Luckily(?), I'm running master, so all the functions are there, and it's 
good to know that there is a programmatic way to make it work, however 
in my case i need to provide a static search string (to pass to an 
external site), so that only certain records get returned to their xml 
parser and displayed.



BTW just yesterday we thought about adding a special index for a similar
searching need, auhtor-at-institute, where a user would type:

ellis@CERN

and the query would return all records where 700 $a contained `ellis'
with the corresponding 700 $u containing `cern'.

So your email is coming very timely :)

P.S. The post-filtering technique could be generalised into a new
  second-order search operator, say @, so that all the following
  queries would be legal:

 author:ellis@affiliation:CERN
 author:ellis@relator:editor
 title:of@provenance:arXiv
Wow! The second-order search operation is exactly what i need! This is 
indeed a very powerful option to have around!
(just imagine being able to use the same idea to create a n-th order 
search operator!)


Thanks for the prompt reply,
Best regards,
Theodoros


ps. I suppose that this enhancement will require a lot of new code in 
several modules... Let's hope that I'll be able to patch my installation 
if/when committed to master...

Re: External collection items not counted in the parent collection in websearch

2013-12-02 Thread Theodoros Theodoropoulos


Thanks Nikos,

I applied it already and I can verify that it's working!

On 2/12/2013 9:58 ??, Nikos Kasioumis wrote:
A patch has been provided for this issue 
http://invenio-software.org/ticket/1651 and it has been merged in  
http://invenio-software.org/repo/invenio/commit/?id=6be1b3d522ae94383710fed1d29ae8122ac21b8f.

How to find records with fulltexts? (Introduction of fulltextcount index?)

2013-11-28 Thread Theodoros Theodoropoulos


Hello everyone,

I'm looking for an easy way to explain to a librarian how to get a list 
of records that have (-not) fulltext(s). (I remember something similar 
being discussed a while ago with Ferran and Alexander).
Searching for: 8564_:[CFG_SITE_URL]* gives me all records with local 
urls, but it's not very pretty. [btw, replacing 8564_ with fulltext 
gives no results]. Maybe a 'special' fulltextcount index (like 
authorcount) would be useful... Having said that, I suppose I would need 
a custom tokenizer for that, no?


Also, if possible, please add a hint in the search-guide help page (in 
the span-queries paragraph maybe?) about ways to check the existence (or 
not) of a marc (sub)field, as this seems to be a common request by 
librarians.

Mostly for my reference, such an example would be:
Show records that have ie a year field:
and year:0-Z
Show records that DO NOT have ie a year field:
and not year:$-~

(this trick doesn't work for the 'fulltext' index. 8564_ should be used 
instead.)


Best regards,
Theodoros

External collection items not counted in the parent collection in websearch

2013-11-26 Thread Theodoros Theodoropoulos


  
  
Hello all,

I'm not sure if anyone actively uses externally hosted collections,
but I verified that items in such collections are not counted in the
parent collection(s) in websearch. I'm not interested in nested
external collections (I think that this is impossible anyway), I
just want to have a proper sum of records even if the (first) child
collection is external.

See the attached image for an example. The 'external' server is also
an invenio instance and everything else works properly.



Is it possible to make this work? Should I create a ticket?

Cheers,
Theodoros

Re: Bibupload/FFT only, replacing Marc records

2013-11-15 Thread Theodoros Theodoropoulos

Thank you Sam for the clarification (btw, I 'fixed' my records by just 
re-uploading[1] the marc, so no problem there).
Bibdocfile is a powerful module and definitely my best friend in this 
case, however, I indeed find bibupload documentation a bit misleading on 
this matter.


As bibupload is a module widely used by people and that this issue could 
potentially lead to temporary loss of marc data (you don't always have 
the xml of the records to 'fix' them and reverting all of them into a 
previous revision is a bit 'tricky'), I will submit a ticket (of 
'trivial' priority) so that -when time for the documentation updates 
comes- you also update the relevant part of the -otherwise clear and 
extensive- bibupload documentation.


Cheers,
Theodoros


[1] Using bibedit list-revisions followed by --revert-to-revision for 
these records would be a 'cleaner' alternative, but I also found the 
opportunity to do some trivial batch changes to the marc, so reuploading 
was a better solution in this case.


On 15/11/2013 10:01 μμ, Samuele Kaplun wrote:

In data venerdì 15 novembre 2013 18:14:57, Theodoropoulos Theodoros ha
scritto:

Hello everyone,

Today I run a bibupload -n -r xxx.xml job with only FFT tags in order to
enrich some of my records with fulltexts. Everything went fine (no
errors/no warnings) but the affected records got replaced entrirely by 3
tags (001/005/856) wiping out everything else.

Shouldn't 'replace' mode (when used only with FFT tags) be smart enough and
not touch (replace) the bibs? Reading the bibupload docs, I believed that I
was allowed to use 'replace' mode with files...

Is this a known behavior (probably even fixed in a personal branch), or did
I hit another bug? ;)

Hi!

It's a feature ;-) not a bug (OK maybe it's a bug in the documentation)?
bibupload --replace is really a replace, no matter what you think: if you pass
an empty record with FFTs it will drop the existing records (and all the
bibdocs attached) and replace with the empty record, executing the FFTs.

If you want to just drop the files you can use bibdocfile CLI first.

Alternatively you can send a bibupload --correct (with empty record, thus
corrcting nothing, and FFTs, thus correcting corresponding bibdocs.)

Cheers!
Sam

Important change in recent shibd.conf that breaks invenio/shibboleth interaction

2013-11-13 Thread Theodoros Theodoropoulos

I would like to share an issue I experienced today and is likely to come 
up with any new invenio/shibboleth installations.


It seems the in recent shibboleth rpms (at least 2.5.x ones) there is a 
new directive that goes by default into shibd.conf and breaks how 
invenio interacts with the shibd deamon.


The 'suspicious' new code in shibd.conf is the following:

Location /Shibboleth.sso
  Satisfy Any
  Allow from all
/Location

A simple test to replicate the problem (provided that you have at least 
set up invenio-apache-vhost-ssl.conf to use shibboleth) is to check for 
a link that should -normally- be handled by shibboleth. For example: 
/Shibboleth.sso/Login


The symptom is that the relevant part of the 
invenio-apache-vhost-ssl.conf is overridden for the /Shibboleth.sso/ 
urls and hence not handled properly... Thus, instead of logging in, you 
get a 404 error from Invenio.


The solution is to remove (or comment-out) the relevant 4 lines from 
shibd.conf and restart apache.


In case you verify my findings, you might want to put a note in the 
HowToIntegrateWithShibboleth wiki entry...


Best regards,
Theodoros Theodoropoulos


ps. Although it's simple to understand once you see it, it took me ~4 
hours to rule out all other possible shibboleth/invenio/wsgi/etc 
configuration problems and start looking at the basic apache 
configurations line-by-line...

Re: Exception in bibrank (citerank_citation_t) with empty citations table


Hello Tibor,


Are there really no citations to read?

(1) If you don't have any citations on your system, then this simply
means that the ranking daemon is not ready for a situation with zero
citations.  We used to have such a problem in the past:

http://invenio-software.org/ticket/499

Maybe it is back.  In this case you may want to inactivate this ranking
method until we check and fix.
I'm pretty sure that there are no citations in my system (at least not 
at this point).
So, if you test in a dummy Invenio site with a few demo records with no 
citations (the Poetry collection maybe?) and verify my findings, it 
might be a bug.
Having said that, I won't be needing citation-based ranking in the near 
future (or at least until I put some useful citation data), and I would 
be happy to inactivate it, but how?


Cheers,
Theodoros

BibIndex VERY slow for itemcount and filetype indexes and Atlas records


(Following Sam's suggestion I'm re-sending this to the dev list.)

I'm currently experiencing VERY slow bibindex times (~2-3min/rec) for 
itemcount and filetype indexes[1] and records that have many authors 
(for example Atlas related experiments). I'm running the latest master. 
Have you noticed it too at CERN?


I understand that -as Esteban mentioned- these indexes are very complex 
and they are computed on-the-fly, but still the time spent on them makes 
the re/indexing unusable.
I could always re-enable CFG_BIBUPLOAD_SERIALIZE_RECORD_STRUCTURE and 
maybe speed things up a litte, but I was trying to save some space in 
the DB :)


I don't want to delete the index, in order to be on par with other 
installations and I only have ~200 such records, but as  you understand 
it takes several hours to index them! (If you experience this issue as 
well, with all these experiments going on, it will take forever to 
re/index INSPIRE regardless of all the computing power you put on it...).


Is there anything that I could do to make it run a bit faster?

I was also wondering... while you're optimizing the algorithms (and only 
if bibfield is not used elsewhere) whether I could tweak (i.e. empy) 
bibfield.cfg to speed things up.


Cheers,
Theodoros


[1] The itemcount index is bearable (maybe because I'm not using 
circulation and hence I don't have items?), the filetype takes forever...

Re: Exception in bibrank (citerank_citation_t) with empty citations table


On 12/11/2013 5:17 μμ, Tibor Simko wrote:


The UI way is to use the BibRank Admin Interface:

https://localhost/admin/bibrank/bibrankadmin.py

there is an option to `Delete' a ranking method.
Oh, I was disabling citation and citerank_citation_t (for all my 
collections) but I suppose this isn't enough... I will try delete.

I'm a newbie in BibRank and I never really touched the default settings...

As an additional feedback, bibrank -S produces the following:

2013-11-12 18:15:27 -- Running rank method: times cited.
2013-11-12 18:15:27 -- No new records added since last time method was run
2013-11-12 18:15:27 -- Showing statistic for selected method
2013-11-12 18:15:27 -- Method name: times cited
2013-11-12 18:15:27 -- Short name: citation
2013-11-12 18:15:27 -- Last run: 2013-11-12 08:06:54
2013-11-12 18:15:27 -- Number of records: 10
2013-11-12 18:15:27 -- Lowest value: 1 - Number of records: 5
2013-11-12 18:15:27 -- Highest value: ('', -99) - Number of records: 0
2013-11-12 18:15:27 -- Divided into 10 sets:
2013-11-12 18:15:27 --
Exception caught: can only concatenate tuple (not int) to tuple
2013-11-12 18:15:27 --
Exception caught: can only concatenate tuple (not int) to tuple
2013-11-12 18:15:27 -- Traceback (most recent call last):
  File 
/usr/lib64/python2.6/site-packages/invenio/bibrank_tag_based_indexer.py, line 
382, in bibrank_engine
rank_method_code_statistics(rank_method_code)
  File 
/usr/lib64/python2.6/site-packages/invenio/bibrank_tag_based_indexer.py, line 
309, in rank_method_code_statistics
lower = -1.0 + ((float(max + 1) / 10)) * (i - 1)
TypeError: can only concatenate tuple (not int) to tuple
2013-11-12 18:15:27 --
Exception caught:
2013-11-12 18:15:27 --
Exception caught:
2013-11-12 18:15:27 -- Traceback (most recent call last):
  File /usr/lib64/python2.6/site-packages/invenio/bibrank.py, line 163, in 
task_run_core
func_object(key)
  File 
/usr/lib64/python2.6/site-packages/invenio/bibrank_tag_based_indexer.py, line 
472, in citation
return bibrank_engine(run)
  File 
/usr/lib64/python2.6/site-packages/invenio/bibrank_tag_based_indexer.py, line 
398, in bibrank_engine
raise StandardError
StandardError
2013-11-12 18:15:27 -- Task #158 finished. [ERROR]

But should(?) probably not worry you too much as it might be related to 
the fact that bibrank did not finish properly...

Re: Exception in bibrank (citerank_citation_t) with empty citations table


On 12/11/2013 6:26 μμ, Alessio Deiana wrote:
It is complaining that you have no citations indexed by bibrank. 
Therefore ranking by citation

won’t work.
Did you run bibrank -w citation ? Or maybe your local records don’t 
have any valid ref/cit
Well... this is my point. That in case no citations are present (which 
applies to my records), this exception pops ups...
This is the first time bibrank is run and I didn't specify any 
particular ranking method (so citation was executed too).


I now DELETED (as opposed to disabling them for all my collections) 
citerank_citation_t, citerank_pagerank_c and citerank_pagerank_t methods 
(keeping backups of the cfg files) and the remaining ranking methods 
seem to be executed smoothly (fingers crossed).

Re: argument of type 'NoneType' is not iterable for filetype/itemcount indexes and deleted records [SOLVED]

2013-11-11 Thread Theodoros Theodoropoulos


Thanks Esteban, with the patch the [re]indexing works :)

Cheers,
Theodoros

On 11/11/2013 3:23 μμ, Esteban J. G. Gabancho wrote:
I was able to reproduce the error in my machine I found what was 
cousing it, I atach you a very little patch.


Once you apply it could you please run:

inveniocfg --load-bibfield-conf
dbexec  echo DELETE FROM bibfmt WHERE format='recjson'

Cheers,
Esteban.

Error when reverting a deleted record from Bibedit (gui)

2013-11-11 Thread Theodoros Theodoropoulos

In bibedit, i tried to revert back a DELETED record and got the 
following error:


2013-11-11 15:07:57 -- Task #131 started.
2013-11-11 15:07:57 -- Input file 
'/opt/invenio/var/tmp-shared/bibedit-cache/bibedit_record_2_1.xml', input mode 
'replace'.
2013-11-11 15:07:57 --  -ERROR: Missing 005 -
2013-11-11 15:07:57 -- Traceback (most recent call last):
  File /usr/lib64/python2.6/site-packages/invenio/bibtask.py, line 984, in 
_task_run
if callable(task_run_fnc) and task_run_fnc():
  File /usr/lib64/python2.6/site-packages/invenio/bibupload.py, line 2912, in 
task_run_core
results_for_callback=results_for_callback)
  File /usr/lib64/python2.6/site-packages/invenio/bibupload.py, line 2837, in 
bibupload_records
tmp_vers = tmp_vers)
  File /usr/lib64/python2.6/site-packages/invenio/bibupload.py, line 332, in 
bibupload
submit_ticket_for_holding_pen(rec_id, err, Missing 005. Inserting record into 
holding pen.)
  File /usr/lib64/python2.6/site-packages/invenio/bibupload.py, line 741, in 
submit_ticket_for_holding_pen
bibcatalog_system.ticket_submit(subject=%s: %s by %s % (msg, rec_id, 
user), recordid=rec_id, text=text, queue=CFG_BIBUPLOAD_CONFLICTING_REVISION_TICKET_QUEUE, 
owner=uid)
  File /usr/lib64/python2.6/site-packages/invenio/bibcatalog_system_email.py, 
line 88, in ticket_submit
ownerset =  owner: + escape_shell_arg(owner) + '\n'
  File /usr/lib64/python2.6/site-packages/invenio/shellutils.py, line 298, in 
escape_shell_arg
raise TypeError(msg)
TypeError: ERROR: escape_shell_arg() expected string argument but got '1L' of type 
'type 'long''.
2013-11-11 15:07:57 -- Task #131 finished. [CERROR]


FYI, /opt/invenio/var/tmp-shared/bibedit-cache/bibedit_record_2_1.xml 
DOES have a 005 field and is:

controlfield tag=00520130514094613.0/controlfield

Any ideas?

Cheers,
Theodoros

Exception in bibrank (citerank_citation_t) with empty citations table

2013-11-11 Thread Theodoros Theodoropoulos


Hello everyone,

It seems that when there are no citations in a site's records, the bibrank job
(more specifically the citerank_citation_t method) produces an exception and 
the whole
bibrank job stops. This is part of the relevant log:

2013-11-12 08:11:49 -- size of reversedict 10
2013-11-12 08:11:49 -- size of citationdict 10
2013-11-12 08:11:49 -- size of selfcitedbydict 10
2013-11-12 08:11:49 -- size of selfcitdict 10
2013-11-12 08:11:49 -- Total time of get_citation_weight(): 237.47 sec
2013-11-12 08:11:49 -- No need to update the indexes for citations.
2013-11-12 08:11:49 -- Running rank method: citerank_citation_t
2013-11-12 08:11:49 -- Error while extracting citation data from 
rnkCITATIONDATA table
2013-11-12 08:11:49 -- Error: No citations to read!
2013-11-12 08:11:49 -- Error: No citations to read!
2013-11-12 08:11:49 -- Traceback (most recent call last):
  File /usr/lib64/python2.6/site-packages/invenio/bibtask.py, line 984, in 
_task_run
if callable(task_run_fnc) and task_run_fnc():
  File /usr/lib64/python2.6/site-packages/invenio/bibrank.py, line 163, in 
task_run_core
func_object(key)
  File 
/usr/lib64/python2.6/site-packages/invenio/bibrank_citerank_indexer.py, line 
782, in citerank
raise Exception
Exception

FYI, I have not changed the default citerank_citation_t.cfg

Is this something known and maybe(?) fixed in a personal branch or should I 
create a ticket?

Cheers,
Theodoros Theodoropoulos

Re: bibcatalog_system_email exception in BibEdit

2013-11-07 Thread Theodoros Theodoropoulos

Only for historical reasons (and while waiting for the 'INSPIRE 
mega-branch' soon-to-be-merged to master), I managed to easily revert 
this behavior by commenting out

createReq({recID: gRecID, requestType: 'getTickets'}, onGetTicketsSuccess);
in bibedit_engine.js

Ok, it's not really a 'fix', but it's the best one can do while the 
relevant function is not implemented in bibcatalog_system_email.py :)


Cheers,
Theodoros

On 4/11/2013 11:26 πμ, Samuele Kaplun wrote:

Hi Theodoros,

In data lunedì 4 novembre 2013 11:23:20, Theodoros Theodoropoulos ha scritto:

It seems that in the master branch (at least a few commits back) if one
tries to edit any record (even from the demo site), the following
exception is thrown:

uri: /record/edit/?

** Traceback details

Traceback (most recent call last):
File
/usr/lib64/python2.6/site-packages/invenio/webinterface_handler_wsgi.py,
line 507, in application ret = invenio_handler(req)
File
/usr/lib64/python2.6/site-packages/invenio/webinterface_handler.py, line
362, in _profiler return _handler(req)
File
/usr/lib64/python2.6/site-packages/invenio/webinterface_handler.py, line
424, in _handler return root._traverse(req, path, False, guest_p)
File
/usr/lib64/python2.6/site-packages/invenio/webinterface_handler.py, line
239, in _traverse return obj._traverse(req, path, do_head, guest_p)
File
/usr/lib64/python2.6/site-packages/invenio/webinterface_handler.py, line
252, in _traverse result = _check_result(req, obj(req, form))
File
/usr/lib64/python2.6/site-packages/invenio/bibedit_webinterface.py, line
144, in index json_data))
File /usr/lib64/python2.6/site-packages/invenio/bibedit_engine.py, line
406, in perform_request_ajax
response.update(perform_request_bibcatalog(request_type, recid, uid)) File
/usr/lib64/python2.6/site-packages/invenio/bibedit_engine.py, line 1289,
in perform_request_bibcatalog status=['new', 'open'], recordid=recid)
File
/usr/lib64/python2.6/site-packages/invenio/bibcatalog_system_email.py,
line 58, in ticket_search raise NotImplementedError
NotImplementedError

btw, BibCatalog configuration in invenio-local.conf has the default
values and I'm not using RT (so CFG_BIBCATALOG_SYSTEM = EMAIL and so
on). I'm not using any new/fancy stuff, so I wonder why the
NonImplementedError...

Should I open a ticket, did I misconfigure something or should I simply
wait for this feature to be implemented?

I'm running v1.1.2.513-53f4 (a few commits back, but I will try to
compile and use the latest one).
I know that master branch is a bit experimental (an at some points not
very advisable to put into production), but I will be needing
authorities in this system, so maint-1.1 was not an option and I cannot
wait for 1.2... Would you consider the current master a bit too
risky/unstable to use in production?

We discovered this bug also when applying master branch to the INSPIRE
project. A fix is coming soon...

Cheers!
Sam

bibcatalog_system_email exception in BibEdit

2013-11-04 Thread Theodoros Theodoropoulos


Good morning,

It seems that in the master branch (at least a few commits back) if one 
tries to edit any record (even from the demo site), the following 
exception is thrown:


uri: /record/edit/?

** Traceback details

Traceback (most recent call last):
  File 
/usr/lib64/python2.6/site-packages/invenio/webinterface_handler_wsgi.py, line 
507, in application
ret = invenio_handler(req)
  File /usr/lib64/python2.6/site-packages/invenio/webinterface_handler.py, 
line 362, in _profiler
return _handler(req)
  File /usr/lib64/python2.6/site-packages/invenio/webinterface_handler.py, 
line 424, in _handler
return root._traverse(req, path, False, guest_p)
  File /usr/lib64/python2.6/site-packages/invenio/webinterface_handler.py, 
line 239, in _traverse
return obj._traverse(req, path, do_head, guest_p)
  File /usr/lib64/python2.6/site-packages/invenio/webinterface_handler.py, 
line 252, in _traverse
result = _check_result(req, obj(req, form))
  File /usr/lib64/python2.6/site-packages/invenio/bibedit_webinterface.py, 
line 144, in index
json_data))
  File /usr/lib64/python2.6/site-packages/invenio/bibedit_engine.py, line 
406, in perform_request_ajax
response.update(perform_request_bibcatalog(request_type, recid, uid))
  File /usr/lib64/python2.6/site-packages/invenio/bibedit_engine.py, line 
1289, in perform_request_bibcatalog
status=['new', 'open'], recordid=recid)
  File /usr/lib64/python2.6/site-packages/invenio/bibcatalog_system_email.py, 
line 58, in ticket_search
raise NotImplementedError
NotImplementedError

btw, BibCatalog configuration in invenio-local.conf has the default 
values and I'm not using RT (so CFG_BIBCATALOG_SYSTEM = EMAIL and so 
on). I'm not using any new/fancy stuff, so I wonder why the 
NonImplementedError...


Should I open a ticket, did I misconfigure something or should I simply 
wait for this feature to be implemented?


I'm running v1.1.2.513-53f4 (a few commits back, but I will try to 
compile and use the latest one).
I know that master branch is a bit experimental (an at some points not 
very advisable to put into production), but I will be needing 
authorities in this system, so maint-1.1 was not an option and I cannot 
wait for 1.2... Would you consider the current master a bit too 
risky/unstable to use in production?


Best regards,
Theodoros Theodoropoulos

invenio_2013_08_22_hstRECORD_affected_fields.py minor issue

2013-11-04 Thread Theodoros Theodoropoulos


invenio_2013_08_22_hstRECORD_affected_fields upgrade recipe executes:

ALTER TABLE hstRECORD ADD COLUMN affected_fields text NOT NULL default 
'' AFTER job_details


The upgrade script gives a warning that BLOB and TEXT columns cannot 
have DEFAULT values.


Although probably not important to the table's functionality, you could 
try removing/changing default '' to suppress the warning...


Cheers,
Theodoros Theodoropoulos

Re: bibupload: ERROR: Invalid Revision (with 'fake' record replacement procedure)

2013-11-01 Thread Theodoros Theodoropoulos


On 31/10/2013 5:46 μμ, Samuele Kaplun wrote:



During a migration phase to a system running the latest 1.1.x and using
the usual recipe [copy bibrec IDs along with creation/modification times
between systems and then 'replace'(insert) records with bibupload -n -r
xxx.xml], I get the following error:

   -ERROR: Invalid Revision - 'INVALID REVISION : 20130930185930 for
Record 250733 not in Archive.'

This is kind of strange as the revision should always ends with a .0...
Hmmm... I verified that the xml contains a 'proper' 005 (that ends with 
a .0). Also, the code in bibupload_revisionverifier.py seems correct. 
Maybe it's just a display issue? If so, this does the trick:


--- /opt/invenio/lib/python/invenio/bibupload_revisionverifier.py   
2013-10-21 08:01:29.463652013 +0300
+++ /opt/invenio/lib/python/invenio/bibupload_revisionverifier.py.test  
2013-11-01 08:59:28.510219744 +0200
@@ -361,7 +361,7 @@
 r_date = upload_rev.split('.')[0]

 if r_date not in [k[1] for k in get_record_revisions(self.rec_id)]:
-raise InvenioBibUploadInvalidRevisionError(self.rec_id, r_date)
+raise InvenioBibUploadInvalidRevisionError(self.rec_id, 
upload_rev)
 else:
 raise InvenioBibUploadMissing005Error(self.rec_id)


This check obviously relates to tag 005 and did not exist in previous
releases. Is there a flag to override this behavior and have the records
replaced (inserted) anyway? (Or should i put all 005s in some table
prior to import?)

Nope. And actually it's strange that you have a revision that does not exist
in the history of records...
Well, In this particular case, its normal. See, that I'm only 'faking' 
bibrecs by just manually creating entries in bibrec. Nothing more. And 
the idea is to upload records from another invenio instance, so that 
timestamps and bibrec IDs are the same across systems (I didn't bother 
messing around with hstRECORD, as I thought bibupload would update it 
during import).



Indeed dropping the 005 would do the trick...

Indeed. Tried it and it works.


But if you can ticketize this with more details we can work on a real solution
:-)

I would be happy to, but it might be nothing wrong here...
Unless of course we're talking about a feature request for a new flag in 
bibupload (ie --skip-005) to properly handle (skip) 005s in input 
records... If you're in to this, you could implement a whole set of 
'skips' (like --skip-001) or -even better- a more generic 
--skip-tag=XXX,YYY. Sounds interesting?



Cheers,
Theodoros

Re: Lost with LibreOffice configuration

2013-10-31 Thread Theodoros Theodoropoulos


On 31/10/2013 11:42 πμ, Samuele Kaplun wrote:

What you can do as a quick workaround is to add right before the:

[...]
sys.path.append(/opt/invenio/lib/python)
from invenio.websubmit_file_converter import CFG_OPENOFFICE_TMPDIR
[...]

But of course! (Silly me...)
I also had to add: sys.path.append(/usr/lib64/python2.6/site-packages) 
for the remaining modules that are needed

Hopefully this should be enough. If the LibreOffice Python and the regular
Python have two different major version they will though keep on recompiling
the .pyc files that inveniounoconv needs :-(
LibO python is 2.6.1 and system python is 2.6.6. I suppose that this 
will not pose an extra problem here, right?


FYI, /opt/invenio/bin/inveniocfg --check-openoffice now returns Ok :)

Thank you Sam for helping me!

Re: bibupload: ERROR: Invalid Revision (with 'fake' record replacement procedure)

2013-10-31 Thread Theodoros Theodoropoulos


Commenting my own mail:

I suppose i could remove 005s completely, but since revisions between 
the two systems are identical, I was seeking an easy way to put 005s as 
well.


If not possible/advisable/easy to do, I'll skip them :)

T. T.

Lost with LibreOffice configuration

2013-10-30 Thread Theodoros Theodoropoulos


Good evening,

I happened to compile invenio with LibreOffice 4.x 
(--with-openoffice-python=/opt/libreoffice4.0/program/python) but it 
seems that versions 4.x contain python 3.x, and thus 
/opt/invenio/bin/inveniounoconv cannot run properly (there are important 
changes in print becoming print() and in exceptions).

(I now remember hitting that same wall in the past too, but...)

The question is:
What do I have to do to 'tell' invenio to use LibreOffice 3.6 python 
WITHOUT COMPILING EVERYTHING from source again? (don't ask...)


What I tried already:
I changed CFG_PATH_OPENOFFICE_PYTHON = 
/opt/libreoffice3.6/program/python inside invenio-autotools.conf

and rerun inveniocfg --update-all
Now inveniocfg --list conains the correct OFFICE_PYTHON path.

FYI, I also changed the 1st line in /opt/invenio/bin/inveniounoconv to: 
#!/opt/libreoffice3.6/program/python


But when executing /opt/invenio/bin/inveniounoconv, I STILL get the 
following error:

Traceback (most recent call last):
  File /opt/invenio/bin/inveniounoconv, line 50, in module
from invenio.websubmit_file_converter import CFG_OPENOFFICE_TMPDIR
ImportError: No module named invenio.websubmit_file_converter

(ipython produces no error for the same line)

I even tried using CFG_PATH_OPENOFFICE_PYTHON = /usr/bin/python 
(because invenio/websubmit_file_converter.py exists only in system's 
site-packages) the  but I see the same error.


I'm a bit lost here :(


Best regards,
Theodoropoulos Theodoros

Re: Questions regarding Knowledge Bases [again]

2013-07-23 Thread Theodoros Theodoropoulos


Thanks Jerome,
Your comments were indeed very helpful. I was confusing the notions of 
'reciprocal' and 'search' but your examples made it all clear [1]


I'm still unable to use KBs in websubmit template files (although the 
same function works in bibconvert) and I was wondering what might be wrong.
Seeing an older Tibor presentation, I was left to believe that KBs work 
with the websubmit module. So, where does the module look for the 
relevant files? I've tried /opt/invenio/var/www/admin/bibconvert/, 
/opt/invenio/etc/bibconvert/config/ (the dir where the .tpl files 
reside), the database (BibKnowledge), but no luck :(


I fear the solution is somewhere under my nose, but I cannot find it...

Best regards,
Theodoros

[1] Having said that, I seem to recall that the first element in a KB 
works like a 'key'. Therefore, looking into your example, the 3rd line 
could(?) produce some 'strange' results, no?

A---B
B---A
A---C (hmm...)
DEF---GHI

Access restrictions in Invenio's OAI-PMH service

2013-07-04 Thread Theodoros Theodoropoulos


Hello everyone,

I was recently re-configuring my Invenio oai repository configuration 
and the got the following idea regarding managing access in the oai2d 
service:
Since responses to requests sent to the OAI service are formatted as 
standard HTTP responses (see: 
http://www.openarchives.org/OAI/openarchivesprotocol.html#HTTPResponseFormat), 
a firerole(-like) access list could be applied to this service, so when 
requests come from certain IP ranges, the service could work as 
expected, or else an 403 error could be returned. For this, a new 
webaccess role/action(like 'accessoaiservice') could be created, so that 
all relevant libraries can refer to it.


If one wants to spend more time on it, a web gui could be created in the 
oairepository admin page, that -in addition to the firerole support- 
could allow a set of specific user/password pairs to access the service. 
To take it even further, these access restrictions could(?) be applied 
per OAI collection, allowing different people/IPs to access different 
sets of setSpecs.


If you think that it might be useful to others and could be implemented 
in a future release, I would be happy to create a relevant 'enhancement 
request' ticket, so that it does not get forgotten.


Kind regards,
Theodoros Theodoropoulos

Is_Original_Submitter issue with webaccess submit role for MBI

2013-06-10 Thread Theodoros Theodoropoulos


Hi to all,

I'm creating some custom websubmit functions to handle local needs when 
it comes to ability to Modify an existing record (similar to 
Is_Original_Submitter) and I came across the following issue:


In order for one to get the Modify existing record button in submit 
screen, he/she must have access to MBI for that doctype.
But then, when Is_Original_Submitter is executed, the check (auth_code, 
auth_message) = acc_authorize_action(user_info, 
submit,verbose=0,doctype=doctype, act=act) will always return 
auth_code=0 (Authorization granted).
Which means that ALL users who can see the Modify existing record 
button will be able to make the modification because they have 
*explicit* power to do so.


So, how can one get this power only because he/she is the original 
submitter?
An idea would be that the user does not have the the MBI for this 
doctype, but gets special permission for a specific record from 
Is_Original_Submitter. But then how the user can hit the MBI button? 
It's not shown...
Even if I put it as a static link with all the required parameters, a 
big warning will popup saying You are not authorized to perform this 
action. Try to login with another account. and Is_Original_Submitter 
will get no change to get executed and give them the special permission 
for this record...


Am I missing something obvious?

The only solution I see is to give everyone who has SBI, MBI permissions 
as well, and IN ADDITION run checks to see if a user is the original 
submitter before allowing him/her to perform modifications to the 
record. But then a new action (or role?) that will act as a 
modify-superadmin will be required that will allow for record 
modifications even if 'original submitter' checks are not satisfied...
Is this the way to go? If so, addition to the default action(/roles ?) 
and changes to Is_Original_Submitter websubmit function might be needed 
for the demo site.


Best regards,
Theodoros

Re: Copyright notice per docfile

2013-06-03 Thread Theodoros Theodoropoulos

Although I vaguely remember hearing something about it in the past, I 
have a simple question related to this topic[1]:

Can we get 'See/See also' related results using KBs?

For example, I have:
100 $0 ID:1 $a Lastname, Firstname
and in another record:
100 $0 ID:1 $a SimilarLastname, SimilarFirstname
(note the SAME ID for the two names!)

What I want is to get results for SimilarLastname too when I search for 
Lastname.


[btw, mapping: 1---Lastnameand Lastname---SimilarLastname
Seems not a proper option because we lose the 'connection' that 
SimilarLastname has also ID1 and maybe other IDs may have 
SimilarLastname. Also, even with that I'm not sure how I can apply both 
these KBs in a search query]


If this is impossible with current implementation using KBs, then I 
believe it should be noted as another VERY useful thing that will come 
along with Authorities in Invenio.


Cheers,
Theodoros


[1] You realize that we keep discussing this very important issue of 
Authorities in a thread with an irrelevant subject :) Let's hope that 
we'll manage to dig these replies up in a future search for this topic...

Re: Copyright notice per docfile

On 28/5/2013 8:02 μμ, Tibor Simko wrote:

Depending on the Invenio version, the BibDoc objects have `more info'
property store that basically serve as a storage for any key-value
combination. We use it e.g. to store technical metadata about pictures:
width, height, position on page. It could be used to store per-docfile
license/copyright information.

Alternatively, in the past, there was also an approach to enrich BibDoc
independently with licensing information and then represent it in MARC
accordingly. We did not commit this part, but we may perhaps revive it
with Jerome.
Licensing Information per docfile is a useful thing to have in Invenio.
Having said that, using a seperate 'field' in bibdocfile -like comment
or description- to keep it (although one possible way to do it) will
probably require a lot of changes in several files.

Did you opt for some solution in the meantime? It would be nice to
settle on a technique and offer it out-of-the-box with Invenio demo
submissions. I think the former technique would be preferable.
I had to come up with a solution really quick with the invenio code that
was available two months ago.
So, i decided to add the license in bibdocfile's comment field in the
form: License:by-sa. For that, I'm using a heavily modified version of
Upload_Files websubmit function.
I have also written/edited relevant format templates/output formats to
'decode' this data and display the license images (currently only used
for CC licenses and only if comment field begins with license:) and
link to the detailed license url.
I'm attaching 3 small screenshots in case you want to take a sneak
preview of the output result. The filenames are _really_ dummy (i'm now
ashamed of them) and the general idea was to include the most peculiar
example with a lot of attachments and file versions/file formats, etc.
The only known drawback I'm having with my approach is that the the tab
'Files' does NOT show any Licensing information. I know this is
important, BUT in order to implement it, I would have to make a lot of
changes in several core Invenio functions (in definitions, return values
etc) and I wanted to avoid that as much as possible, hoping that a more
proper/complete solution would come at some point from you :)

Best regards,
Theodoros
attachment: Invenio-CCLicencesInDetRec.pngattachment: Invenio-CCLicencesInMiniPanel.pngattachment: Invenio-CCLicencesInBriefRec.png

Re: Question about document migration to new server


On 29/5/2013 10:34 πμ, Tibor Simko wrote:


That's generally very good way of thinking.  In this concrete use case,
the trouble is that the Invenio upload API does not allow to submit a
file with a given wanted docID.  (Unlike submitting a record with a
given wanted recID, which is possible.)  So the proper answer to
Theodoros's need would be to extend Invenio's FFT API support to allow
forcing of docIDs, for example.  However, this would take time.  Hence
the need for a low-level DB dump/load solution that we have been
discussing in this thread.

(BTW another `proper' solution could be simply not to reuse the old
docIDs, but load the files anew via FFT, keeping only recIDs the same.
The docIDs are never exposed to end users, so this might be acceptable.
However, some massaging would have to be done a posteriori anyway, in
order to port old logs and stats information.  Hence the first solution,
more advantageous in my eyes.)
True. I also mentioned this, more 'proper' alternative to move bibdocs, 
and I would be happy with it, if and only if the bibdoc STATS would be 
exported along with each docfile. Since this is currently not available, 
I would have to separately move the rnkDOWNLOADS and change the bibdoc 
ids from old to new. I decided that this in not a good idea (mainly 
because I would have to alter millions of lines, and also because I 
would be messing with statistics and it would be difficult for me to 
find out if something went wrong in the process). So I too believe that 
keeping the same bibdoc id's is the way to go in this particular case, 
and it will also make it easier for me to check things between old and 
new systems.


Having said all that, It would be a nice addition for the 'future' if an 
admin would be able to export the bibdocs along with their stats. Also, 
why not even be able to import them back to the system... I see possible 
issues with adding/replacing bibdocs (and what will happen to their 
statistics), but I'm just mentioning an issue that I personally needed 
once. In case you see a more generic Invenio community need for that, 
I'm sure you will find a way (or make the necessary changes in DB) to 
overcome that.


Best regards,
Theodoros

Re: Unexpected keyword argument errors in invenio.err for doctypeconfiguresubmissionfunctions and doctypeconfiguresubmissionpages


On 28/5/2013 7:50 μμ, Tibor Simko wrote:

On Mon, 15 Apr 2013, Theodoros Theodoropoulos wrote:

I'm pretty sure it will be reproducible in a stock Invenio site
too. Can you verify it so that I submit a ticket?

Verified.  Please submit a new `known issue' for WebSubmit.  Thanks for
reporting!

Done. Ticketized under http://invenio-software.org/ticket/1521
(btw I cannot select 'known issue' for a ticket... One must probably 
have higher credentials for that :) )


Best regards,
Theodoros

Re: Increase in maximum allowed size in Document Type ID, Element Name


On 29/5/2013 12:53 μμ, Tibor Simko wrote:

That said, we are however fully concentrating on developing the new
WebDeposit module which brings many new goodies and which will
eventually replace WebSubmit.  So, if at all possible, it would be
simpler if you worked with the current WebSubmit field length
limitations until WebDeposit comes?
Well, I'm several months past this point of having to choose the Element 
names :)
I ended up using ugly abbreviations, but I'm happy I didn't create any 
extra potential problems for a future migration from WebSubmit to 
WebDeposit.


Having said that, I will be happily surprised if/when such an 
'automatic' migration procedure to WebDeposit will be made available 
(bearing in mind all the custom elements, checks, javascript-insertion 
dummy elements, etc WebSubmit users might have introduced).


Cheers,
Theodoros

Re: Copyright notice per docfile


Hey, I could give you my kidney to have you revive BibAuthority!!

I've written many times to some developers and to the list regarding 
BibAuthority, but I never got a straight answer regarding to it's status.
Nevertheless, I based all my author tags based on the very promising 
'recipe' found in https://twiki.cern.ch/twiki/bin/view/CDS/BibAuthority 
hoping that some day Invenio would get some proper authority support.


I don't know how useful it would be for licensing, but I can guarantee 
that it will make the life of MANY Invenio users way better. I remember 
such a strong request in the last IUGM. Please tell me that you're 
thinking seriously to revive BibAuthority! (even if it's a lie...)


Best regards,
Theodoros

Question about document migration to new server

2013-05-28 Thread Theodoros Theodoropoulos


Hi to all,

I want to move a collection (with it's documents) to a new server, while 
retaining the record ids and all the STATS.


Things are somewhat easy with bibrecs. However, there are issues with 
the handling of bibdocs.
With the FFT tag way of copying bibdocs, I cannot find a way to force a 
bibdoc to get a specific bibdoc_id (id in bibdoc table), so the only way 
I see as an alternative, is the manual copy of selected files (with 
their full path structure in the filesystem) along with the copy of the 
related bibdoc, rnkDOWNLOADS, rnkPAGEVIEWS entries.
Do you see any drawbacks with this method? Should I pay special 
attention to something? Do you have a better idea (ie 'translation' of 
bibdoc ids in bibdoc, rnkDOWNLOADS, rnkPAGEVIEWS to match the new ones 
created by FFT)?


Your ideas are most welcome!

Best regards,
Theodoros

Re: search-guide.webdoc with Greek translation added

2013-05-28 Thread Theodoros Theodoropoulos


On Τρίτη, 28 Μάιος 2013 4:10:32 μμ, Tibor Simko wrote:


I'd like to acknowledge the co-authorship of Vaggelis in the commit log.
Is this OK, and if so, is it possible to know and publicly publish his
email address?
Of course! It's Evangelos Savvidis (vsavi...@gmail.com) and you may 
publish both the name and email.


Thank you!

Re: Question about document migration to new server

2013-05-28 Thread Theodoros Theodoropoulos


On 28/5/2013 3:23 μμ, Tibor Simko wrote:
Yes, I think this is the fastest way to go. Don't forget all bibdoc 
related tables such as `bibdocmoreinfo'. 
Whoops! I didn't see that coming! I'm moving data away from an old 
invenio (0.99.x series) and there are no such tables as 
bibdocmoreinfo/bibdocsinfo
Any chance they can be regenerated from the filesystem? (with bibdocfile 
--fix-bibdocfsinfo-cache or --fix-marc or fix-all)?


Theodoros

Questions regarding Knowledge Bases

2013-05-23 Thread Theodoros Theodoropoulos


Hello everyone,

Just some quick questions this time regarding Knowledge Bases :)

First of all, most KBs are now moved into the DB (and served via 
BibKnowledge). That's great, but are these KBs available to template 
(.tpl) files used to convert/create record data? Or for that purpose 
will I have to use flat files? Which tools have access ONLY to flat or 
DB KBs?


I'm also having some problems understanding the difference between 
search/match/search (in KB)/search (reciprocal) in the following part of 
bibconvert admin guide:


{*input_value---output_value*}

KB(file,1) searches the exact value passed.
KB(file,0) searches the KB code inside the value passed.
KB(file,2) as 0 but not case sensitive
KB(file,R) replacements are applied on substrings/characters only.

bibconvert look-up value in KB_file in one of following modes:
===
1 - case sensitive / match (default)
2 - not case sensitive / search
3 - case sensitive / search
4 - not case sensitive / match
5 - case sensitive / search (in KB)
6 - not case sensitive / search (in KB)
7 - case sensitive / search (reciprocal)
8 - not case sensitive / search (reciprocal)
9 - replace by _DEFAULT_ only
R - not case sensitive / search (reciprocal) replace

If i want to covert between A-B (and vice versa) and I have a 
translate.kb containing A---B, will KB(translate.kb,8) will be enough 
for both-ways translation?


Thanks in advance for your time,
Theodoros

Re: Integration of scholar into Invenio 1.0Rc0

2013-05-15 Thread Theodoros Theodoropoulos

Hello everyone,

I think I just managed to backport it to 0.99.x as well... At least the
new META tags have appeared :)

The situation is a bit more complicated when compared to 1.0.x series,
since additions to other files is required (for example htmlutils.py) as
well as changes to the bfe_meta.py in order to make it work with the old
series...

If there is anyone needing this feature in an old 0.99.x installation, I
could provide some hints (nothing advanced to be honest), but I fear
that with this king of 'hacking' -especially in htmlutils.py-, internal
parts might have been affected, so I strongly discourage it for
productions systems [1].

Best regards,
Theodoros

[1] Having said that, I personally plan to keep it for a day or two
(always keeping my fingers crossed) and in the first glimpse of
suspicious behavior, I will revert back to the original files.

On 15/5/2013 3:59 μμ, Jerome Caffaro wrote:

Hi Greg,

Gregory Favre wrote:

I know you have updated a lot of things helping the interaction with this
service. Basically what was done?

One thing that was done was the integration of some Google Scholar
markup in the page header. The following commit shows which files
have been touched:
http://invenio-software.org/repo/invenio/commit/?id=96ac7c5eefb647de22dd41c4baba0d7e37597abb

In two words, a new output format HDM has been added, which is
called by the search engine when displaying a record. Note that
these files have been later enhanced by further commits.

Does it seam feasible to backport these features from invenio 1.1 to 1.0rc0?

I don't foresee any major trouble if you try to backport the necessary
files to your installation. It is mostly new files, with very
few changes to an existing one.

Best regards
--
Jerome Caffaro

Re: Record Editor showing Checking status... and nothing more

2013-05-13 Thread Theodoros Theodoropoulos


Hello Petr,

I too can verify the issue (tried with the latest maint-1.1 branch).

The problem lies not in invenio sources, but in the latest 
jquery.hotkeys.js!


If one runs make install-jquery-plugins, he/she gets the latest 
version of jquery.hotkeys.js (size 3598 bytes) that says:

/*
 * One small change is: now keys are passed by object { keys: '...' }
 * Might be useful, when you want to pass some other data to your handler
 */

Probably this is related to the issue that you mention.

Luckily, I had a previous version of jquery.hotkeys.js (size 3080 
bytes), copied it in the /opt/invenio/var/www/js/ directory and 
everything went back to normal :)
As a quick-n-dirty solution, you can try the patch that is included in 
the end of the email in order to revert to the previous version of 
jquery.hotkeys.js, but the Invenio devs will probably come up with a 
better solution...


Best regards,
Theodoros Theodoropoulos


ps. This is the patch to revert to an old version of jquery.hotkeys.js 
that works (until a more permanent fix is produced)...


--- /opt/invenio/var/www/js/jquery.hotkeys.js 2013-05-13 10:11:23.825329769 
+0300
+++ /opt/invenio/var/www/js/jquery.hotkeys.js.orig  2013-05-12 
15:58:33.742961433 +0300
@@ -10,6 +10,11 @@
  * Binny V A, http://www.openjs.com/scripts/events/keyboard_shortcuts/
 */

+/*
+ * One small change is: now keys are passed by object { keys: '...' }
+ * Might be useful, when you want to pass some other data to your handler
+ */
+
 (function(jQuery){

jQuery.hotkeys = {
@@ -22,7 +27,8 @@
96: 0, 97: 1, 98: 2, 99: 3, 100: 4, 101: 5, 102: 6, 
103: 7,
104: 8, 105: 9, 106: *, 107: +, 109: -, 110: ., 111 : 
/,
112: f1, 113: f2, 114: f3, 115: f4, 116: f5, 117: f6, 118: 
f7, 119: f8,
-   120: f9, 121: f10, 122: f11, 123: f12, 144: numlock, 145: 
scroll, 191: /, 224: meta
+   120: f9, 121: f10, 122: f11, 123: f12, 144: numlock, 145: 
scroll, 186: ;, 191: /,
+   220: \\, 222: ', 224: meta
},

shiftNums: {
@@ -33,18 +39,23 @@
};

function keyHandler( handleObj ) {
+   if ( typeof handleObj.data === string ) {
+   handleObj.data = { keys: handleObj.data };
+   }
+
// Only care when a possible input has been specified
-   if ( typeof handleObj.data !== string ) {
+   if ( !handleObj.data.keys || typeof handleObj.data.keys !== 
string ) {
return;
}
-
+
var origHandler = handleObj.handler,
-   keys = handleObj.data.toLowerCase().split( );
+   keys = handleObj.data.keys.toLowerCase().split( ),
+   textAcceptingInputTypes = [text, password, number, email, url, range, date, month, 
week, time, datetime, datetime-local, search, color];

handleObj.handler = function( event ) {
// Don't fire in text-accepting inputs that we didn't 
directly bind to
if ( this !== event.target  (/textarea|select/i.test( 
event.target.nodeName ) ||
-event.target.type === text) ) {
+   jQuery.inArray(event.target.type, 
textAcceptingInputTypes)  -1 ) ) {
return;
}

Re: Access rights for files

2013-04-29 Thread Theodoros Theodoropoulos




So we encourage all our users to upload a full text and only give access
at the library if we are allowed to. So this might be very well two
distinct tasks.
Your scenario is not uncommon i think... And it seems not that difficult 
to implement!
Even with the existing Create_Upload_Files_Interface, you could define a 
custom 'static' restriction -as Sam said- that allows librarian group 
and submitter and denies everyone else. IF user chooses to 'restrict' 
the document this will be applied. The added bonus is that with invenio 
1.1 you could CHANGE this restriction afterwards with Document File 
Management. OR, even with earlier versions you could probably implement 
the same functionality with Submit Revised Version ('SRV') action.

Re: Access rights for files

2013-04-29 Thread Theodoros Theodoropoulos


On 29/4/2013 9:37 πμ, Alexander Wagner wrote:

## Eg:
## [('', 'No restriction'), ('restr', 'Restricted')]
CFG_BIBDOCFILE_DOCUMENT_FILE_MANAGER_RESTRICTIONS = [
 ('', 'Public'),
 ('restricted', 'Restricted')]


If this could learn something like 6-month embargo and 12-month
embargo e.g. by reading a field like Lars' OpenAIRE setting we would
already have a 99% solution and the rest could be done easily by hand.
It would fall back to some time calculations, but well. Probably one
could merge it with the 942-aproach of OpenAIRE?
This type of embargo is what we also needed in the past. Back in the 
0.99.x days I had it solved -partly- with a custom Upload_Files.py 
websumbit function, but nowadays you could do even more with a custom 
Websubmit Element of type Response. You can check Upload_Files 
websubmit element in the demo site where you may call the 
create_file_upload_interface python function with all the parameters you 
need.
Since you could calculate the exact date in a 12 months-from-now 
embargo restriction, you could create a semi-dynamic embargo firerole 
restriction in the response element! The downside is that you could not 
easily edit the end of embargo date at a later stage using the gui. But 
it's a start...

Sam could correct me if I'm wrong...

Best regards,
Theodoros

Re: Access rights for files

2013-04-26 Thread Theodoros Theodoropoulos


Hello Lars and Alexander!

I too would be interested in such a functionality!

Personally, I don't think there is an easy way to do it.

As i see it, a new Action (similar to SRV) should be created (along 
with relevant websubmit functions and elements) that would read and 
display the current files along with their restrictions.
Important note: an update to relevant bibdocfile.py functions would be 
needed as 'get-restrictions' is not supported!


If you feed the current restrictions to a websubmit form's textarea 
that is editable, it's easy to ask the bibdocfile library to 're-set' 
the permissions for each bibdocfile to the new values when the 
librarian clicks submit.


The current Create_Upload_Files_Interface.py functions does not include 
such functionality, but it could be done... (Having said that, one 
sould probably have to mess with websubmit_templates.py too)


I'm using a custom websubmit_function (based on the 'older' 
Upload_Files) to perform a similar task but during the initial phase of 
the submission only where 'set' is only needed. If one needs to 'edit' 
the relevant data he/she has to contact me so that I would manually run 
the bibdocfile (in cli) with the new values.


Just my two cents,
Theodoros

Re: Access rights for files

2013-04-26 Thread Theodoros Theodoropoulos


Hello Ludmila,

On 26/4/2013 5:51 μμ, Ludmila Marian wrote:
The restrictions are stored as the status of the file, so get_status() 
should do the trick to get the restriction of the file.
True, true! I see that you're already using  get_status to merge 
restrictions when merging bibdocs.
My suggestion for a 'get_restriction' function was based on the already 
implemented get_comment, get_description, get_version, etc functions.
The functionality as you pointed out is there, the new name would just 
be more in par with the rest of the function names :)


In any case, thanks for the hint!
Theodoros

Re: Exceptions due to attacks

2013-04-25 Thread Theodoros Theodoropoulos


On 25/4/2013 12:29 μμ, Theodoros Theodoropoulos wrote:

I tried the same with cds.lib.auth.gr and it also displays a 404 error
(i don't know if an error is logged)

Correction: I meant cds.cern.ch :)

Re: Exceptions due to attacks

2013-04-25 Thread Theodoros Theodoropoulos


On 25/4/2013 12:37 μμ, Ferran Jorba wrote:

but try an index.php or any other missing hit at http://cds.cern.ch.  It
is effectively handled by Invenio.
My point exactly. I see that both my installations and CERN's correctly 
handle those 'attacks'. I even tried with .php and .py files and there 
is no exception raised and sent to the admin even if you set 
CFG_SITE_ADMIN_EMAIL_EXCEPTIONS = 2 in invenio(-local).conf


Unless I'm missing something here, I suspect something weird happening 
only with your installation...


Theodoros

Unexpected keyword argument errors in invenio.err for doctypeconfiguresubmissionfunctions and doctypeconfiguresubmissionpages

2013-04-15 Thread Theodoros Theodoropoulos


Hello all,

If you go to WebSubmit Admin-[any demo doctype]-[any action]-view functions (or 
view interface)

you get the following error shown in invenio.err (no exception is shown to the 
frontend):

=

Wrong GET parameter set in calling a legacy publisher handler for 
doctypeconfiguresubmissionfunctions: expected_args=['ln', 'addfunctionscore', 
'addfunctionstep', 'addfunctionname', 'configuresubmissionaddfunctioncommit', 
'configuresubmissionaddfunction', 'deletefunctionscore', 'deletefunctionstep', 
'deletefunctionname', 'movetofunctionscore', 'movetofunctionstep', 
'movetofunctionname', 'movefromfunctionscore', 'movefromfunctionstep', 
'movefromfunctionname', 'movedownfunctionscore', 'movedownfunctionstep', 
'movedownfunctionname', 'moveupfunctionscore', 'moveupfunctionstep', 
'moveupfunctionname', 'action', 'doctype', 'req'], found_args=['action', 
'doctype', 'viewSubmissionFunctions']

* 2013-04-15 20:57:40 - TypeError: doctypeconfiguresubmissionfunctions() got 
an unexpected keyword argument 'viewSubmissionFunctions' 
(webinterface_handler_wsgi.py:677:mp_legacy_publisher)

** User details
 agent: Mozilla/5.0 (Windows NT 6.1; WOW64; 
rv:18.0) Gecko/20100101 Firefox/18.0
 email: th...@xxx.yyy.zzz
 group: []
 guest: 0
  language: en
  login_method: Local
  nickname: admin
precached_canseehiddenmarctags: True
precached_permitted_restricted_collections: ['Theses', 'Drafts', 'ALEPH 
Theses', 'ALEPH Internal Notes', 'ISOLDE Internal Notes', 'Atlantis Times 
Drafts']
precached_sendcomments: True
precached_useadmin: True
   precached_usealerts: True
  precached_useapprove: True
  precached_usebaskets: True
   precached_usegroups: True
precached_useloans: True
 precached_usemessages: True
 precached_usepaperattribution: True
   precached_usepaperclaim: True
precached_usestats: True
   precached_viewclaimlink: True
 precached_viewsubmissions: True
   referer: 
   remote_host:
 remote_ip: www.xxx.yyy.zzz
   session: cc9766caae02814486f3c9048aebd64a
   uid: 1
   uri: 
/person/S.D.Ellis.1?open_claim=True

** Traceback details

Traceback (most recent call last):
  File 
/usr/lib64/python2.6/site-packages/invenio/webinterface_handler_wsgi.py, line 
677, in mp_legacy_publisher
return _check_result(req, module_globals[possible_handler](req, **form))
TypeError: doctypeconfiguresubmissionfunctions() got an unexpected keyword 
argument 'viewSubmissionFunctions'

** Stack frame details

Frame mp_legacy_publisher in 
/usr/lib64/python2.6/site-packages/invenio/webinterface_handler_wsgi.py at line 
685
---
   682 expected_defaults = list(inspected_args[3])
   683 expected_args.reverse()
   684 expected_defaults.reverse()
  685 register_exception(req=req, prefix=Wrong GET parameter 
set in calling a legacy publisher handler for %s: expected_args=%s, found_args=%s % 
(possible_handler, repr(expected_args), repr(req.form.keys())), alert_admin=CFG_DEVEL_SITE)
   686 cleaned_form = {}
   687 for index, arg in enumerate(expected_args):
   688 if arg == 'req':
---
  inspected_args =  ArgSpec(args=['req', 'doctype', 'action', 
'moveupfunctionname', 'moveupfunctionstep', 'moveupfunctionscore', 
'movedownfunctionname', 'movedownfunctionstep', 'movedownfunctionscore', 
'movefromfunctionname', 'movefromfunctionstep', 'movefromfunctionscore', 
'movetofunctionname', 'movetofunctionstep', 'movetofunctionscore', 
'deletefunctionname', 'deletefunctionstep', 'deletefunctionscore', 
'configuresubmissionaddfunction', 'configuresubmissionaddfunctioncommit', 
'addfunctionname', 'addfunctionstep' [...]
   expected_defaults =  ['en', '', '', '', '', '', '', '', '', '', '', '', 
'', '', '', '', '', '', '', '', '', '', '']
 err =  'TypeError(doctypeconfiguresubmissionfunctions() 
got an unexpected keyword argument \'viewSubmissionFunctions\',)'
form =  {'action': 'SBI', 'doctype': 'DEMOART', 
'viewSubmissionFunctions': 'view functions'}
  the_module =  '\'# -*- coding: utf-8

Copyright notice per docfile

2013-04-10 Thread Theodoros Theodoropoulos


Hello everyone,

I was wondering what would be the best approach to use in order to 
insert a small copyright notice or a copyright id (ie 'by-nc-sa') for 
each docfile during file submission. I'm not concerned -for the moment- 
with how to implement it in a form, or how to display it in the 
bfe_fulltext, but where this could should be kept in the docfile related 
structure.


Currently, bibdocfile supports 'description' and 'comment'. Description 
could be an option, but it should better be kept for other things. 
Comment looks more promising, but i cannot see where it's currently 
used/displayed. Do you think it's an appropriate attribute to set and 
use for copyright, or are you planning to set a new attribute for 
copyright notice in the future?


Thanks for your time,
Best regards,
Theodoros


ps. I've discovered a weird behavior in bibdocfile where 
--set-description and --set-comment on a record that has many docfiles 
applies only to the last docfile. Is this an implementation decision or 
is it a bug? (I thought I should ask before submitting a bug ticket :) )

Re: Copyright notice per docfile [update]

2013-04-10 Thread Theodoros Theodoropoulos


On 10/4/2013 4:35 μμ, Theodoros Theodoropoulos wrote:
ps. I've discovered a weird behavior in bibdocfile where 
--set-description and --set-comment on a record that has many docfiles 
applies only to the last docfile. Is this an implementation decision 
or is it a bug? (I thought I should ask before submitting a bug ticket 
:) )
It seems that this has been fixed in the current 1.1-maint branch! I was 
testing this with a much older revision... :blush:

Increase in maximum allowed size in Document Type ID, Element Name

2013-03-27 Thread Theodoros Theodoropoulos


Hello everyone,

Currently, maximum Document Type ID size is 10 chars and maximum 
Element Name size is 15 chars. I vaguely remember that these limits 
were even lower in older versions of Invenio (cdsware). Unless there is 
a design reason (ie big and 'ugly' parameters in URLs or unnecessary 
waste in DB tables), I believe that these limits are somewhat 
...limiting and they should probably be increased.
If current Invenio admins/developers think that this could be useful to 
others and easy to implement, I could submit a relevant enhancement 
request ticket.
If, on the other hand, this is not needed to the vast majority and could 
be 'fixed' by simply altering the max allowed size in a DB column, I 
could proceed with the change in my local installation (provided that 
this won't break how things work).


Best regards,
Theodoros

Internal server error in invenio-demo-next [only with Greek translation]

2013-03-06 Thread Theodoros Theodoropoulos


Dear devs,

While playing around with invenio-demo-next.ch, I found that an 
Internal Server Error is produced if you select the Greek translation 
and try to browse/search any collection. This error does not appear for 
other translations...
I know next is under heavy development, but I thought I should mention 
it just in case it rings any bells.


Best regards,
Theodoros

Show records locked by users/queues (along with user/queue details) in bibeditcli

2013-02-15 Thread Theodoros Theodoropoulos


Hello everyone,

Some times I happened to need the list of locked bibs (along with user 
details) and although bibedit_utils.py already has all the necessary 
functions, there is no such command line switch in bibedit/bibeditcli.


So I thought that some useful new options that could be implemented 
would be:
- show user_details for a specific record_id  [probably using 
record_locked_by_user_details(recid, uid)]
- show all locked record ids (by queues/users), maybe even filtered by 
dates (so as to see which are locked for t long, although if 
CFG_BIBEDIT_TIMEOUT is set properly, this might not be needed).

- 'release' certain record_id (s) (also deleting the related cache)

Although probably easy to implement, all these are of minor importance, 
and might even not fall into your TODO list, so I'm not submitting an 
enhancement ticket. Having said that, if you see any real use for 
Invenio users, I would be happy to see them being added in a future 
release:)


Best regards,
Theodoros

Exception with bibauthorid --update-personid

2013-02-13 Thread Theodoros Theodoropoulos


Hello everyone,

I've downloaded and compiled the latest maint-1.1 from git because I 
wanted to test the latest improvements in the bibauthorID module that 
seem very interesting. After running all the required steps, creating 
the demo site and loading the demo-records, I run:

sudo -u apache /opt/invenio/bin/bibauthorid --update-personid

and got the following exception:

2013-02-13 13:17:13 -- Entering task_sleep_now_if_required with 
status=UNKNOWN

[...] repeated 100 times [...]
2013-02-13 13:17:13 -- Entering task_sleep_now_if_required with 
status=UNKNOWN

  File /opt/invenio/bin/bibauthorid, line 35, in module
main()
  File /opt/invenio/lib/python/invenio/bibauthorid_cli.py, line 35, 
in main

daemon.bibauthorid_daemon()
  File /opt/invenio/lib/python/invenio/bibauthorid_daemon.py, line 
104, in bibauthorid_daemon

task_run_fnc=_task_run_core)
  File /opt/invenio/lib/python/invenio/bibtask.py, line 480, in task_init
ret = _task_run(task_run_fnc)
  File /opt/invenio/lib/python/invenio/bibtask.py, line 962, in _task_run
if callable(task_run_fnc) and task_run_fnc():
  File /opt/invenio/lib/python/invenio/bibauthorid_daemon.py, line 
147, in _task_run_core

run_rabbit(record_ids, all_records)
  File /opt/invenio/lib/python/invenio/bibauthorid_daemon.py, line 
286, in run_rabbit
rabbit_with_log(None, True, 'bibauthorid_daemon, update_personid on 
all papers')
  File /opt/invenio/lib/python/invenio/bibauthorid_daemon.py, line 
264, in rabbit_with_log
insert_user_log('daemon', '-1', action, 'bibsched', 'status', 
comment=log_comment, timestamp=starting_time)
  File 
/usr/lib64/python2.6/site-packages/invenio/bibauthorid_dbinterface.py, 
line 863, in insert_user_log

action, tag, value, comment))
  File /usr/lib64/python2.6/site-packages/invenio/dbquery.py, line 
211, in run_sql

rc = cur.execute(sql, param)
  File /usr/lib64/python2.6/site-packages/MySQLdb/cursors.py, line 
203, in execute

if not self._defer_warnings: self._warning_check()
  File /usr/lib64/python2.6/site-packages/MySQLdb/cursors.py, line 
117, in _warning_check

warn(w[-1], self.Warning, 3)
  File /usr/lib64/python2.6/site-packages/invenio/errorlib.py, line 
569, in fun

traceback.print_stack()
/usr/lib64/python2.6/site-packages/MySQLdb/cursors.py:203: Warning: 
Incorrect integer value: '' for column 'userid' at row 1

  if not self._defer_warnings: self._warning_check()
2013-02-13 13:17:14 -- Task #15 finished. [DONE]

sudo -u apache /opt/invenio/bin/bibauthorid --disambiguate also gives 
the following exception:

2013-02-13 13:44:15 -- Task #16 submitted.
-bash-4.1# sudo -u apache /opt/invenio/bin/bibauthorid 16
2013-02-13 13:44:19 -- Task #16 started.
2013-02-13 13:44:22 -- Traceback (most recent call last):
  File /opt/invenio/lib/python/invenio/bibtask.py, line 962, in _task_run
if callable(task_run_fnc) and task_run_fnc():
  File /opt/invenio/lib/python/invenio/bibauthorid_daemon.py, line 
152, in _task_run_core

run_tortoise(bool(bibtask.task_get_option(from_scratch)))
  File /opt/invenio/lib/python/invenio/bibauthorid_daemon.py, line 
305, in run_tortoise

tortoise(modified)
  File 
/usr/lib64/python2.6/site-packages/invenio/bibauthorid_tortoise.py, 
line 124, in tortoise

assert all(stat == os.EX_OK for stat in exit_statuses)
AssertionError
2013-02-13 13:44:22 -- Task #16 finished. [CERROR]

Can you verify/reproduce these exceptions? (There is always the 
possibility that I might have broken something while testing things).
BTW, the aid* tables are filled with the appropriate author data and 
|aidUSERINPUTLOG| contains only the following row (1, 0, '2013-02-13 
13:17:13', 0, 'daemon', -1, 'PID_UPDATE', 'bibsched', 'status', 
'bibauthorid_daemon, update_personid on all papers')


Thanks for your time,
Theodoropoulos Theodoros


ps. And one bonus question regarding BibAuthorID module :) Is there any 
way for a user to search for authors using any of their external IDs?

Bibcheck cli?

2013-01-29 Thread Theodoros Theodoropoulos


Hello everyone,

I need to import a few thousand records (that were converted to marcxml 
from RefWorks format using bibconvert) into my Invenio instance.
Because of the complexity of the original records -and mainly because 
not all users followed the instructions when filling the Refworks forms- 
the final records may lack some important fields to us (like 100$a or 
260$c).
So, i was wondering if there is way to check the produced marcxml file 
before importing it into the system and bibcheck came to my mind... It 
seems however, that although bibcheck web admin is there, there is no 
bibcheck cli... Is this module still in use? I also remember vaguely 
that they were plans to add checking functions in bibedit (web/cli)... 
With these in mind, do you have any suggestion as to what is currently 
the best way to check records that exist either in a marcxml file or in 
the holding pen?


Thanks in advance for your time,
Theodoros

Re: Bibcheck cli?

2013-01-29 Thread Theodoros Theodoropoulos


Thank you Sam for the prompt reply!

No problem, I'll go with python. I just wanted to double check, in 
order to see if I was missing something that was already there and did 
the trick.

(pseudo)authorities: what is supported in Invenio/Inspire?

2012-11-30 Thread Theodoros Theodoropoulos


Good morning to all,

In order to be able to plan ahead in my installation, I would like to 
know if there is any way to deal with authorities in Invenio.


The BibAuthority module (described in detail here: 
https://twiki.cern.ch/twiki/bin/view/CDS/BibAuthority) is EXACLY what i 
would like to have, but i cannot find it in my installation nor in the 
invenio/inspire repo. Was the module abandoned/postponed/replaced by 
something even better?


On the other hand, inspirehep.net seems to have such functionality 
(HepNames, Institutions, Conferences) seem to be 'authority' records 
that are considered when performing a search.


Could you please shed a light on what's being available now and what's 
planned for the near future?


Best regards,
Theodoropoulos Theodoros


ps. I apologize for sending a similar mail in the past too. It's quite 
urgent that i know at least some general information and the best 
practices for the current/next Invenio version in order to make the 
right decisions for my next milestone. I'll do the 
reading/implementation/testing myself.

Re: Add all search results to a basket?

2012-11-30 Thread Theodoros Theodoropoulos


On 30/11/2012 10:58 ??, Lars Holm Nielsen wrote:

Hi,

Would it be all records for an entire search or just all records 
displayed on one result page? In case of the latter, there's a toggle 
all button in the next-branch (see example on: 
http://invenio-demo-next.cern.ch/search?p=action_search=).


Cheers,
Lars

Great demo site! It looks beautiful and is definitely user-friendlier 
(and very google-like) ;)

Is that faceted search on the left? Hooray! Can't wait!

Is this based on a new webstyle template that can be chosen in 
invenio(-local).conf or will this be the default look-and-feel of the 
next version?


You never stop surprising us (in a good way)! Keep up the great work,
Theodoros

Re: (pseudo)authorities: what is supported in Invenio/Inspire?

2012-11-30 Thread Theodoros Theodoropoulos


Hello Alexander,

It's been a while since we talked at the last Invenio Users Meeting, 
but i'm following the list and I've seen the great work you've done at 
Juelich.



We currently use some sort of authority control in JuSER where we mainly
implemented the functionality within websubmit type of things. This
however does not (yet?) use bibauthority. Unfortunately, the
searchengine does not yet know about authorities. If we can be of some
help here, drop me a note.
I'm too using a way to correctly SELECT a user (with all proper IDs, 
affiliations, etc) from a list in websubmit part. However, the values 
are copied to 100/700__0,a,e,u bib record fields. This ensures 
consistency across records, but it's really a plan-b solution -but the 
only way to do things because authorities are not(?) supported. Plus, I 
think that there is no easy way to use this data in BibEdit: one could 
select from a list of authors in 100/700__a, but i'm not sure that this 
selection could update other subfields of 100/700 accordingly.
Ideally, with authorities, in the bib record, only the 'reference' to 
the authority would be used, so that if anything changes in the author, 
all the 'connected' records will know the change, plus you get see/see 
also fields, alternate/translated names support etc.
That would also involve indexing and searching, and this is described 
nicely, here http://twiki.cern.ch/twiki/bin/view/CDS/BibAuthority




You can check out what we did at http://juser.fz-juelich.de/ in the
Authorities-collections. You might also want to check out the Marc we
used. E.g. for an institute http://juser.fz-juelich.de/record/98380 You
may use the arrow links to go to predecessors, successors or top.
You seem to have it nicely ordered. I'm currently interested mostly on 
authors, but Institutions will come next. I'll check your marc to get 
an idea of what you've used. Thanks again for sharing your work!



We would definitely also want to know how they can be used within
search. I mean not simple stuff like searching for an ID, we have this
already but searching for an ID x and find all records for ID x and y as
the authority record of x list's y as addional identifier. We work
around our current limitations e.g. using the index cid (contributing
institutes also gathering all former forms of the institute) but this is
not really done in a nice way. Bascially a record here just has enough
collection entires, but it's no real authority lookup.

Exactly!


When we started to implement this stuff I checked closely with Annette
Holtkamp, so I think we should indeed use the same or at least a very
similar authority record layout as Inspire.
Hmm.. I didn't go yet too much into Inspire's (pseudo?)authority marc, 
but will do soon. I also believe that we should always try to use 
common fields when possible, keeping always in mind the marc standards.


Having said that, I am mostly interested in more fundamental issues:
- (Is/was/will there be) support for authorities soon 
-indexing/searching/bibedit support included- and what can i do now to 
be ready?
- Is the final module going to be something similar to what's being 
described in that wiki page?

- Is it planned for 1.2 1.x or 2.x series?

My best regards to all,
Theodoros

Case insensitivity in some webaccess related tables vs relevant python functions

2012-11-29 Thread Theodoros Theodoropoulos


Hello everyone,

Yesterday i hit the following situation: My aim was to restrict access 
to collection test to certain users.
Reading the docs, I created a test_viewers role and connected certain 
users to the viewrestrcoll authorization to the test collection (all 
using the web interface). But, instead of entering test in the 
collection argument of the viewrestrcoll, I mistakenly added Test.
The issue I found was that, even though the parameter 'collection' is 
entered and kept in the DB in the case you enter it, you cannot add ie 
test if you have previously added TEST or Test. If you try that 
you get a warning saying sorry, authorization could not be added, it 
probably already exists. This is obviously related to the DB having 
uf8_general_ci collation, so the relevant check returns a row, 
regardless if you search for Test, test, or TEST...
Having said that, some(?) python libraries distinguish arguments passed 
to authorizations in a case-sensitive way (probably because they work 
with IDs). For example,

search_engine.collection_restricted_p('Test')   is true, while
search_engine.collection_restricted_p('test')  is not.
Furthermore the entries in accARGUMENT are never(?) deleted even if you 
delete the appropriate authorization. This means that if one for some 
reason enters a value for keyword in this table, that value in all 
lower/capital letter combinations is reserved for ever...
I don't know if this is the desired operation, but if not, you might 
either consider changing collation (dangerous?) to utf8_bin for certain 
tables (accARGUMENT for example), or alternatively change certain 
functions (like collection_restricted_p) to be case insensitive.


Although my problem is now solved by changing the string in the DB, I 
thought I should raise the issue in case you wanted to deal with it in 
the future :)


Best regards,
Theodoros

Re: Record Editor Suggest values: ctrl-shift-a brings up Addon Manager in Firefox (windows)

2012-11-29 Thread Theodoros Theodoropoulos

It seems that the key combination is hardcoded in bibedit_keys.js, 
which might require a few changes in the way it's called inside 
bibedit_engine in order for one to be able to pass this parameter... 
(It's a pity because the code is so beautiful and concise :( ).


ps. For now, I will change the value into something else, but i think 
it's an issue that eventually should be dealt with, since it affects a 
large percentage of non-linux users. (Btw, it seems that the same 
shortcut exists in Firefox for linux, but -at least in my installation- 
does NOT bring up the addon manager...)

MathJax/LATEX support for titles in hb,hd


Hello everyone,

I'm trying to parse LATEX strings in titles for the first time, so I 
changed the appropriate entries in the invenio-local configuration file 
(mathjax source was already installed), run inveniocfg --update-all, 
added latex_to_html=yes in the appropriate format template, and... 
nothing. Latex is not parsed.


I even tried importing bibformat_utils in ipython and running i.e. print 
bibformat_utils.latex_to_html(monadic $\forall^{1}_{1}$ positive theory 
over large sets ), but I only get the original string.


What am I doing wrong? I KNOW I'm missing something, because this 
feature works properly in the CERN demo site (the same entries in my 
local demo site, that are not parsed).


Any ideas would be welcome!

Best regards,
Theodoros

How to accessing attached files/temporary marcxml record before approving a record?


Hello everyone,

When a document is in the pending directory (ie is waiting for approval) 
how can a librarian access the attached files and/or the dummymarcxml 
that is produced in the form-submission phase in order to verify that 
everything is in order and proceed with the approval?


The record is not yet integrated, so It seems logical that /record/xxx 
and /record/xxx/files are empty ... (although in the 
Send_Approval_Request websubmit function you send an approval email that 
points to the -always empty- /record/xxx/files dir).
I know i can access and include (in the approval mail) the contents of 
the files in the related pending dir for the record so, I suppose I 
could also attach the whole (parsed?) marcxml in the mail, but I was 
looking for a better solution...


Hmmm... What mechanism are you using at CERN for your librarians that 
handle approval requests?

Any ideas/comments would be welcome!

Best regards,
Theodoros

Re: MathJax/LATEX support for titles in hb,hd


What is the result of not using this latex_to_html functionaility?

The same unparsed formula.


I even tried importing bibformat_utils in ipython and running i.e. print
bibformat_utils.latex_to_html(monadic $\forall^{1}_{1}$ positive theory
over large sets ), but I only get the original string.


That's good :-) Infact MathJax expects to find original LaTeX formulas and
render them on the fly using Javascript on the client's browser. Check, e.g.
the source HTML of:

http://invenio-demo.cern.ch/record/80?ln=en

As you will see formulas are nicely there.

I know! That's why I'm puzzled!


How have you installed the MathJax sources? Have you correctly run

$ sudo -u www-data make install-mathjax-plugin
Yes! I just run it again just in case. No difference. Hmm maybe a 
matter of permissions? Apart from 
/opt/invenio/var/www/MathJax/config/MathJax.js (which is 
apache:apache), the rest are root:root. Having said that, there are no 
related warnings/errors in the apache log file...




Do you have any errors in the Javascript console of your browser?

No. I already checked that before sending the first email.

Any other ideas? I don't know what else to check :(

Re: How to accessing attached files/temporary marcxml record before approving a record?


On Δευτέρα, 26 Νοέμβριος 2012 2:43:49 μμ, Samuele Kaplun wrote:
[...]

So we basically don't use at all the pattern given by WebSubmit,
Move_to_Pending, Move_from_Pending.

The drawback of this approach is that a recid is allocated even in case the
document is then rejected but this is a small price to pay.


Thank you Sam for all these details! They are indeed very interesting 
and I'm so sorry to learn them now[1]... The last few weeks I had to 
create 14 new doctypes with many pages/new custom 
elements/websubmit_functions that all use the good-old APP action 
workflow. I'm afraid to touch the process again in fear that i may 
break things (and you know how 'fragile' things are in the websubmit 
module!)


Having said that, the workflow you describe is waay better and more 
flexible than the old one! Losing a recid no. is nothing much compared 
to what you get in return!



Cataloguers/referees can then approve/amend the document using a dedicated
submission workflow that will read the document from the system, and at the
end will simply update the record to reflect the final collection tag in 980.
Hmmm... Are these websubmit functions publicly available, or are they 
'tailored' only for CERN library?


I'm eager to test them out (when I find some spare time...)

Keep up the good work,
Theodoros

--
[1] I had an indication of this because when i tried to access a newly 
created -but not approved- record, i got a nice big RESTRICTED sign

Re: How to accessing attached files/temporary marcxml record before approving a record?


In principle it could be nice to have one websubmit function that take one
parameter which would be the new value to be put in 980, and having such
submission function taking care of preparing marcxml and triggering bibupload,
etc.

I don't know the internals of your functions, but as i understand it, 
there could a workflow where SBI creates -as usual- the xmlmarc and 
puts the correct collection in your 980 (at least my implementation 
takes doctype and categformat and puts them into 980). From there, in 
order to appove it you only have to take out the restriction bit 
(which, i suppose, is easily done with either direct access to the 
appropriate table, or, even better with a temp .xml file and bibupload) 
and a change in sbmAPPROVAL table...
Even if your workflow initially puts all pending docs in a 'hidden' 
collection, you could put the correct doctype in another local field 
(let's say 989), and then when you approve it, just create a temp .xmp 
file and use bibupload in 'correct' mode to replace the old 980 with 
the value(s) from 989.
Another similar approach would be for the approve function to remove a 
'magic' string (in the already complete record) similar to DELETED, 
which, when present, the record is not displayed/counted in the 
respective collections and is accessible only by the librarians who 
have the 'approve' right for that collection...
This way the approve function can be very generic and no parameter 
passing (for the collection code) is needed.


Just my two cents :)

Cheers,
Theodoros

Re: MathJax/LATEX support for titles in hb,hd


Do you have a public URL where we can check? Is there any reference for
MathJax in the HEAD part of your pages? Could it be that you are working
with a custom template which is not including the %{metaheaderadd}s placemark
(where MathJax would be usually included...
Indeed the meta entries [1] in my source, that are similar to what you 
have in your demo site, BUT in my case the   script 
src='/MathJax/MathJax.js' type='text/javascript'/script is missing!
I'm now debugging to see why. I'll update the thread when i find the 
reason.


Best regards,
Theodoros

[1] BTW, the Default HTML meta.bft is 'Not OK'  because format elements 
'META_OPENGRAPH_VIDEO' and 'META_OPENGRAPH_IMAGE' use unknown parameter 
protocol

Re: MathJax/LATEX support for titles in hb,hd