Invenio Developer Forum - signalutils and Flask-Admin - today at 16:45 CEST

[Lars Holm Nielsen]

Hello:

The next Invenio Developer Forum will take place today at 16:45 CEST in
(i) CERN room 31-S-023 and in (ii) Vidyo videoconferencing room at
.

Today, I will show signalutils, Flask-Admin, pidstore which are soon to hit 
next-branch.

Best regards,
Lars

--
Lars Holm Nielsen
Software Engineer

CERN, IT Department, Digital Library Technology Section
Office 513/1-014
Tel: +41 22 76 79182
Cel: +41 76 672 8927




smime.p7s
Description: S/MIME Cryptographic Signature

Re: Copyright notice per docfile

[Tibor Simko]

On Mon, 03 Jun 2013, Theodoros Theodoropoulos wrote:
> What I want is to get results for SimilarLastname too when I search for
> Lastname.

Right now you could get something like this using synonym KBs, see the
docs for CFG_WEBSEARCH_SYNONYM_KBRS and CFG_BIBINDEX_SYNONYM_KBRS.  That
is, this workaround may lead to false positives, as you mentioned.  For
complete solution, one has to wait for BibAuthority.

Best regards
--
Tibor Simko

RE: Copyright notice per docfile

[Wagner, Alexander]

Hello Tibor!

Perfectly agree that complete Authority is better than KBs on steroids. :)

I came to the idea of "doping" a kb automagically from an auth record form 
another direction.

If you consider a large set of authorities (JuSER holds some 64k of them), and 
take into account, that the improvement of authorities are shared authorities 
you may arrive a the point that for X only a certain subset of auth records is 
applicable. For an average user it's best only to show those applicable to 
avoid wrong entries. We have this case already as well and currently have a 
working, but not too fast solution.

First of all, we have horizontal and vertical connections and thus a real tree. 
We need to recurse through the auth records to get the leaves of such a tree 
only, especially if we want to allow a top level criterion for input (say: 
someone types "energy" she should get "solar cells" as a proposal as this is 
the child of a child of "energy"). Then we need to intersect the leaves found 
with some criterion like "not ignored at X" (this is an intellectual criterion, 
so it needs to be tagged manually; e.g. 
http://juser.fz-juelich.de/record/92322/export/hm, marc 751_7 marks this record 
not to show up at DESY).

Finally, we have to do all this on the fly and return a JSON in websubmit. So, 
it also has to be fast.

Currently, we really do the recursion and the intersection on the backend on 
real data with cached JS output formats. It works, it's even fast enough at the 
moment, but one of my ideas to speed it up was to just condense what's needed 
at X in a specific kb. This would drop the recursion and most likely a bunch of 
the intersections (though they are not that critical). Change rate of those 
datasets is something up to "once in 5 years", so a "once every day" type of 
job would work.

At this point it might be worthwile to consider kb doping.

--

Kind regards,

Alexander Wagner
Subject Specialist
Central Library
52425 Juelich

mail : a.wag...@fz-juelich.de
phone: +49 2461 61-1586
Fax  : +49 2461 61-6103
www.fz-juelich.de/zb/DE/zb-fi


From: Tibor Simko [tibor.si...@cern.ch]
Sent: Monday, June 03, 2013 10:51
To: Wagner, Alexander
Cc: Theodoros Theodoropoulos; project-invenio-devel
Subject: Re: Copyright notice per docfile

On Mon, 03 Jun 2013, Wagner, Alexander wrote:
> I think there is / should be a close connection between knwoledge
> bases and authority. It could well be that real authority records end
> up to be a "librarian accessible" version of a knowledge bases, at
> least for simple auth records.

One could consider BibAuthority as a BibKnowledge on steroids.  KBs
allow for simple ``good -> bad'' substitutions, while authority records
offer much more additional values, comments, material on the target
side, that one can take advantage of.  Kind of `str -> str' vs `str ->
dict' thing.

> One idea is, that you can add an auth record and some bibshed tasklet
> condenses them to suitable kb entries. This will work in some cases.

May be cleaner to keep the two modules independent, and let people
choose which one to use depending on the given need.

> At least, I do not see a way how "my librarians" can handle necessary
> changes to kbs at the moment at all.

Yes, that's the steroid use case.  (Or a regular MARC world use case for
authority records, one could say.)  It is better not to force KBs to
handle these cases, rather to improve BibAuthority.

> Invenio can do a lot here but it just lacks some infrastructure ot
> enforce auth control. Our main problem right now is, however, that
> support for this lacks in the search engine.

ChrisD was working on this.  So the planned revival of BibAuthority
shall add appropriate indexing support, search support, deletion/editing
support...

> @Tibor I'd really like to discuss this when I'm @cern

+1

Best regards
--
Tibor Simko




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

RE: Copyright notice per docfile

[Wagner, Alexander]

Theodoros, this is exactly the functionality that we are lacking at the moment 
in search. For our stats it is not an issue, we simply don't search silly 
things like "real names" but only IDs. For our users it would indeed be helpful 
if they could get a handle on the name without the detour through 
Authorities/People.

Anyway, if this would be possible via a kb it would still be important for "my 
librarian" to fill it in. Though she could add a 400 field to the auth set I 
fear kb editing is out of the question. So even if I fear there is a part 
missing at the moment.

--

Kind regards,

Alexander Wagner
Subject Specialist
Central Library
52425 Juelich

mail : a.wag...@fz-juelich.de
phone: +49 2461 61-1586
Fax  : +49 2461 61-6103
www.fz-juelich.de/zb/DE/zb-fi


From: Theodoros Theodoropoulos [th...@physics.auth.gr]
Sent: Monday, June 03, 2013 12:03
To: Tibor Simko
Cc: Wagner, Alexander; project-invenio-devel
Subject: Re: Copyright notice per docfile

Although I vaguely remember hearing something about it in the past, I
have a simple question related to this topic[1]:
Can we get 'See/See also' related results using KBs?

For example, I have:
100 $0 ID:1 $a Lastname, Firstname
and in another record:
100 $0 ID:1 $a SimilarLastname, SimilarFirstname
(note the SAME ID for the two names!)

What I want is to get results for SimilarLastname too when I search for
Lastname.

[btw, mapping: 1---Lastnameand Lastname---SimilarLastname
Seems not a proper option because we lose the 'connection' that
SimilarLastname has also ID1 and maybe other IDs may have
SimilarLastname. Also, even with that I'm not sure how I can apply both
these KBs in a search query]

If this is impossible with current implementation using KBs, then I
believe it should be noted as another VERY useful thing that will come
along with Authorities in Invenio.

Cheers,
Theodoros


[1] You realize that we keep discussing this very important issue of
Authorities in a thread with an irrelevant subject :) Let's hope that
we'll manage to dig these replies up in a future search for this topic...




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Re: Copyright notice per docfile

[Theodoros Theodoropoulos]

Although I vaguely remember hearing something about it in the past, I 
have a simple question related to this topic[1]:

Can we get 'See/See also' related results using KBs?

For example, I have:
100 $0 ID:1 $a Lastname, Firstname
and in another record:
100 $0 ID:1 $a SimilarLastname, SimilarFirstname
(note the SAME ID for the two names!)

What I want is to get results for SimilarLastname too when I search for 
Lastname.


[btw, mapping: 1---Lastnameand Lastname---SimilarLastname
Seems not a proper option because we lose the 'connection' that 
SimilarLastname has also ID1 and maybe other IDs may have 
SimilarLastname. Also, even with that I'm not sure how I can apply both 
these KBs in a search query]


If this is impossible with current implementation using KBs, then I 
believe it should be noted as another VERY useful thing that will come 
along with Authorities in Invenio.


Cheers,
Theodoros


[1] You realize that we keep discussing this very important issue of 
Authorities in a thread with an irrelevant subject :) Let's hope that 
we'll manage to dig these replies up in a future search for this topic...

Re: Copyright notice per docfile

[Tibor Simko]

On Mon, 03 Jun 2013, Wagner, Alexander wrote:
> I think there is / should be a close connection between knwoledge
> bases and authority. It could well be that real authority records end
> up to be a "librarian accessible" version of a knowledge bases, at
> least for simple auth records.

One could consider BibAuthority as a BibKnowledge on steroids.  KBs
allow for simple ``good -> bad'' substitutions, while authority records
offer much more additional values, comments, material on the target
side, that one can take advantage of.  Kind of `str -> str' vs `str ->
dict' thing.

> One idea is, that you can add an auth record and some bibshed tasklet
> condenses them to suitable kb entries. This will work in some cases.

May be cleaner to keep the two modules independent, and let people
choose which one to use depending on the given need.

> At least, I do not see a way how "my librarians" can handle necessary
> changes to kbs at the moment at all.

Yes, that's the steroid use case.  (Or a regular MARC world use case for
authority records, one could say.)  It is better not to force KBs to
handle these cases, rather to improve BibAuthority.

> Invenio can do a lot here but it just lacks some infrastructure ot
> enforce auth control. Our main problem right now is, however, that
> support for this lacks in the search engine.

ChrisD was working on this.  So the planned revival of BibAuthority
shall add appropriate indexing support, search support, deletion/editing
support...

> @Tibor I'd really like to discuss this when I'm @cern

+1

Best regards
--
Tibor Simko

RE: Copyright notice per docfile

[Wagner, Alexander]

Hi!

I think there is / should be a close connection between knwoledge bases and 
authority. It could well be that real authority records end up to be a 
"librarian accessible" version of a knowledge bases, at least for simple auth 
records.

One idea is, that you can add an auth record and some bibshed tasklet condenses 
them to suitable kb entries. This will work in some cases.

Still, I think, there're some "beasts" of auth controled things. E.g. our 
journals are of this kind. We have such information as well when we're talking 
about "authors" and "institutes", and I'm not sure that this maps to kbs /in 
general/. At least, I do not see a way how "my librarians" can handle necessary 
changes to kbs at the moment at all.

On the other hand I think kbs can not handle connected entries. Say you have a 
logical chain of subsequent entries. Institute A get's renamed to Alpha and 
later on is called A again and you have to track this history. Would be 
horizontal connectinos. Same for vertical A is top of a is top of alpha. And 
then connect this with a horizonal logic on each level. We have this for the 
instiutes mentioned, but also grant like entites show this. (Project B has 
several children and is the successor of A, not all children of A have 
sucessors in B and so on.)

I think all this gets really important if you live on stuff that is used for 
some evaluation of projects or institutes and other kinds of "bean counting". 
How much was accomplished by A split by time and so on.

Invenio can do a lot here but it just lacks some infrastructure ot enforce auth 
control. Our main problem right now is, however, that support for this lacks in 
the search engine. But this is due to the fact that we added most of the input 
methods in websubmit.

IMHO we should discuss this a bit more in detail, but it gets a bit difficult 
without good examples and I think "hands on" will help a lot. @Tibor I'd really 
like to discuss this when I'm @cern and as Theodoros mentinoes IUGM should have 
a topic on this. We already did that at the start of our project, as you will 
remember. But right now I think I can come up with quite a lot more details 
than back then and also some solutions and pending issues.

--

Kind regards,

Alexander Wagner
Subject Specialist
Central Library
52425 Juelich

mail : a.wag...@fz-juelich.de
phone: +49 2461 61-1586
Fax  : +49 2461 61-6103
www.fz-juelich.de/zb/DE/zb-fi


From: Tibor Simko [tibor.si...@cern.ch]
Sent: Thursday, May 30, 2013 10:18
To: Theodoros Theodoropoulos
Cc: Wagner, Alexander; project-invenio-devel
Subject: Re: Copyright notice per docfile

On Wed, 29 May 2013, Theodoros Theodoropoulos wrote:
> Please tell me that you're thinking seriously to revive BibAuthority!

Yes, I confirm.  It was in the plans for the past year already, but the
revival got sidetracked due to shift in priorities.  Now it
progressively surfaces back to higher positions in the priority list...

Best regards
--
Tibor Simko




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt