Just a quick addendum:

I would point out the issues I mention below regarding efforts to improved F-measure derived from human efforts don't even begin to address the enormous amount of relevant work from the Text Mining field (various approaches to improving on standard IR sparse matrix term representation, NLP, LSI, text summarization, text categorization, etc.). There is also the work Bob Futrelle has brought up before regarding "hedging" and other issues that very much effect our ability to automatically perform KE/KR/KM on unstructured text. This is not unrelated to work on "speech acts" as applied to formalization of clinical records, of course.

It's not completely clear to me how or whether that specific work can be put to use on this versioning problem.

Cheers,
Bill


On Jan 12, 2007, at 7:40 AM, William Bug wrote:

The IFOMIS work Dirk, Kirsten, and others have cited on referent tracking is definitely important work to review in this light. I'd not been familiar with the model theoretic work Bijan mentions, but clearly that is important.

Werner Ceusters also has a list - a Google list I believe - on referent tracking.

This work - and related work on "speech acts" - is most definitely relevant to this discussion and very specifically is designed to address ABox. As the citations given indicate, most of this work has been done in the clinical domain with a focus on patient records, which was the origin of this thread and would be directly relevant to the Use Case Nigam put out there.

Some of that work has begun to seep into the discussions regarding the sort of GENBANK issues Kei mentioned, but its still really just discussion to my knowledge. As you could tell from the way I couched my description of that problem, clearly referent tracking is a big part of what must be accomodated in that domain as well - both in terms of the actual content and evolution of a record in GENBANK, TrEMBL, etc., as well as the many ways in which researchers link to and reference such records.

Also - the work I was mentioning regarding TBox focussed, highly granular revisions, has been informally discussed by NCBO folks including Chris Mungal, Fabian Neuhaus, Barry and others - again with an eye toward providing reasoning services to support this requirement of the nature of what Bijan, Dirk and others mention below. This is associated with the discussions on this topic amongst both BIRN, OBI, and NCIT participants but has all been very informal so far - AFAIK.

One of the things I would point out regarding the metadata properties I was referring to, is this was really meant to be just a simple, "low hanging fruit" approach to a much more complicated problem. There was not thought given to how one would actually construct automatic means to mediate reasoning on - or even just representing - the evolving semantic graph. The idea was simply - many biomedical ontology development projects have begun to notice the pressing need for version control which appears to be required at a very granular level. Standard source version control systems - e.g., CVS, SVN, etc. - just make the problem worse in my opinion. This is where I'd differ with the point Vipul makes. It's not that there are NO aspects of the software version process relevant to this issue. It's just I believe there are complex issues in this domain - some of which Bijan mentioned - some of which I mention below regarding application the traditional approach to employing CVs for literature annotation - that extend greatly beyond what the common practice in software version control is intended to support. In that domain, highly granular version management has been required, and I believe something like it will be required in the ontology development space as well. Perhaps that's just a qualification and rewording of the point Vipul was trying to make.

SKOS, as I mentioned, does try to absorb some of what has been done on this issue in the A&I/library science world in relation to CV application to the literature annotation process. This has long been recognized in that field as extremely important to the proper curation of a CV/taxonomy/classification scheme/thesaurus. In that domain, if you step back a bit from the details and ask - what is the intended purpose of a CV in that domain - the answer clearly is to improve both precision and recall (F-measure from standard IR) for boolean, term-based queries. Anyone who has used MEDLINE over the years has learned the utility of this approach - and its limitations (the barrage of false positives and unknown number of false negatives that typically still effect query results). There is no doubt just looked at empirically that having the people who are annotating the literature use a CV greatly improves the F- measure of the search system used to mine the resulting inverted indexes. However, I know from time working with the creators of the Biological Abstracts, that it took months of training for the "indexers" to get good at consistently applying CV terms - and a lot of QA/QC was still needed to constantly monitor the output. The reason really comes down to the lack of complete, detailed definitions and lack of a formal, semantic graph really left way too much leeway for indexers, even when a moderate amount of effort was dedicated to incentivizing indexers. Having said that, when highly specific definitions were used, it was found indexers both greatly slowed in the annotation output AND use of CV terms went way down, both of which are really at odds to the intended goal of the process (back to F-measure), which is to provide maximal annotation given according to a CV. Even with this work, BIOSIS (publishers of the Biological Abstracts) and really all the A&I vendors I knew of, still required a huge educational staff that would constantly travel the world providing demos and updates to librarians, so they could be kept informed on how best to use the resulting CV indexes.

It was still clearly an art to maximize F-measure - one that very much depended on quality and structure of the CV/classification scheme/taxonomy, the talents of the indexers applying the CVs in the annotation process and the talents of the info. retrieval experts/librarians in constructing queries. By far the most confounding aspect of this process was the need to alter indexer and searcher practice, as CV changes were introduced - as was of course inevidible - both due to changes in the *world* and changes in *knowledge representation*, as Bijan describes it below. It was partly because of this, that various CV curatorial practices were developed that again are partially represented in SKOS - fields such as "scope notes", "history notes", etc., which all relate to the versioning issue in this context, but, of course, are designed for human consumption and are not particularly useful to KE/KR algorithms.

My sense - as you can see in that OBI Wiki page I cited - is there is a need to provide such curation support in the ontology development process both to address the lexical issues as has been historically done in info. science/library science, as well as to address semantic graph evolution. Both of these requirements arise due both to changes in *world* and QA/QC performed on the KR (changes in *knowledge*). My sense is in providing this first simple step - a shared collection of AnnotationProperties used across the community when building OWL-based ontologies - we provide the structure required to develop software tools to help automate the process. Nothing extending to the complexity of automatic reasoning, but just something to address the need quickly - a structured model for these processes, if you will, that can evolve toward the more complex "referent tracking" and "speech act" formalism. This stop-gap isn't nearly enough to fully address this complex issue, but it should be relatively easy to implement and to put into practice (with a minimal amount of automated support for ontology curators), and if done correctly, should be something that can migrate to the more complex approach later. Providing too complex a strategy for addressing this versioning issue now might prohibitively slow the ontology development process as it is being carried out by various community biomed. ontology development projects.

As you can tell, this is just a suggestion which OBI, BIRNLex, and a few other ontology developers have just begun to implement, so this is most definitely a work-in-progress.

Having a review of the topic, as Vipul suggests, at this stage in the game by the several folks who've provided valuable pointers and feedback, would be a wonderful idea, I think.

Cheers,
Bill


On Jan 12, 2007, at 6:26 AM, Kashyap, Vipul wrote:



Is there any work in the literature related to:

- Defining what and when a version is?
- Do all updates necessarily lead to a new version?
- Is there a utility to instance versioning?

The observation about the utility of knowledge base update and revision is an astute one. IMHO the utility of instance versioning is not clear either.

Just my 2 cents,

---Vipul


-----Original Message-----
From: [EMAIL PROTECTED] [mailto:public-semweb- lifesci-
[EMAIL PROTECTED] On Behalf Of Bijan Parsia
Sent: Friday, January 12, 2007 5:28 AM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]; 'w3c semweb hcls'; public-semweb-lifesci-
[EMAIL PROTECTED]
Subject: Re: Versioning vs Temporal modeling of Patient State


On Jan 12, 2007, at 9:36 AM, [EMAIL PROTECTED] wrote:

Recently I had an interesting conversation with Werner Cuesters,
professor in Bufallo and colleague of Barry Smith. He has some
theory about ontology maintenance and versioning and it considers
both "classes" and "instances". Both can change either because you
made en error, either you view on the world changed, either because
the world changed . It turns out that you can only handle changes
if you know for each change exactly what de reason of the change
was. That reason should be documented in the system.
[snip]

The standard lingo for this is that a change to the knowledge base
due to a change in the *world* is called an *update* whereas a change
in your knowledge base due to a change in *your knowledge* of the
(current static) world is called a *revision*. The locus classicus
for this, IMHO, is:
        <http://citeseer.ist.psu.edu/417296.html>

Following there model theoretic accounts, there is a spate of work
defining reasoning services that compute the updated or revisied
knowledge base given a proposed update or revision. E.g., recently:
        <http://lat.inf.tu-dresden.de/~clu/papers/archive/kr06c.pdf>

The utility of model oriented revision and update for expressive
logics is, IMHO, not fully established, though it is conceptually
useful in my experience. There is, of course, a large chunk of work
on revising (and even updating) belief *bases*, that is, attending
primarily to the *asserted* set of formulae.

Hope this helps.

Cheers,
Bijan.







THE INFORMATION TRANSMITTED IN THIS ELECTRONIC COMMUNICATION IS INTENDED ONLY FOR THE PERSON OR ENTITY TO WHOM IT IS ADDRESSED AND MAY CONTAIN CONFIDENTIAL AND/OR PRIVILEGED MATERIAL. ANY REVIEW, RETRANSMISSION, DISSEMINATION OR OTHER USE OF OR TAKING OF ANY ACTION IN RELIANCE UPON, THIS INFORMATION BY PERSONS OR ENTITIES OTHER THAN THE INTENDED RECIPIENT IS PROHIBITED. IF YOU RECEIVED THIS INFORMATION IN ERROR, PLEASE CONTACT THE SENDER AND THE PRIVACY OFFICER, AND PROPERLY DISPOSE OF THIS INFORMATION.




Bill Bug
Senior Research Analyst/Ontological Engineer

Laboratory for Bioimaging  & Anatomical Informatics
www.neuroterrain.org
Department of Neurobiology & Anatomy
Drexel University College of Medicine
2900 Queen Lane
Philadelphia, PA    19129
215 991 8430 (ph)
610 457 0443 (mobile)
215 843 9367 (fax)


Please Note: I now have a new email - [EMAIL PROTECTED]





Bill Bug
Senior Research Analyst/Ontological Engineer

Laboratory for Bioimaging  & Anatomical Informatics
www.neuroterrain.org
Department of Neurobiology & Anatomy
Drexel University College of Medicine
2900 Queen Lane
Philadelphia, PA    19129
215 991 8430 (ph)
610 457 0443 (mobile)
215 843 9367 (fax)


Please Note: I now have a new email - [EMAIL PROTECTED]




Reply via email to