On Feb 25, 2013, at 8:12 AM, Seth van Hooland svhoo...@ulb.ac.be wrote:
You want to automate the discovery of people, place names and events within a
large corpus of unstructured documents or metadata (e.g. description field)?
Then you might want to use the Named-Entity Recognition (NER) extension for
OpenRefine that has been developed by Multimedia Lab (ELIS — Ghent University
/ iMinds) and MasTIC (Université Libre de Bruxelles).
Yes, named-entity recognition (NER) is fun.
About a year ago I used a different application to do NER against about 100
digitized files. From my blog posting [0]:
name-entity extraction – There was a desire to list the
underlying names, places, and organizations from each text. These
things can put a text into a context for the reader. Are there a
lot of Irish names? Is there a preponderance of place names from
the United States? To accomplish this task and assist in
answering these sorts of questions, a Perl script was written
around the Stanford Named Entity Recognizer. [1] This script
(txt2ner.pl [2]) extracts the entities, looks them up in DBedia, and
saves metadata (abstracts, URLs to images, as well as latitudes
longitudes) describing the entities to a locally defined XML file
for later processing. (See an example. [3]) A CGI script (ner.cgi [4])
was then written to provide a reader-interface to these files.
Once I NER'ed the files and saved the corresponding linked data, I was able
to create a tablet-based interface providing the means for the reader to see
how the words are used in context, but also read a blurb from wikipedia as well
as map places via Google Maps. For example, some extracts from a book called An
adventure With The Apaches [5] but the data is not as clean as I would hope.
The whole thing was a part of a project we called the Catholic Youth Literature
Project. [6]
The ELIS software looks pretty interesting. [7]
Fun with distant reading and NER.
[0] blog postding - http://blogs.nd.edu/emorgan/2012/03/cyl/
[1] Stanford NER - http://nlp.stanford.edu/software/CRF-NER.shtml
[2] txt2ner.pl - http://dh.crc.nd.edu/sandbox/cyl/bin/txt2ner.pl
[3] intermediate XML file -
http://dh.crc.nd.edu/sandbox/cyl/corpus/advicetoirishgir00cusa.ner
[4] CGI script - http://dh.crc.nd.edu/sandbox/cyl/bin/ner-cgi.pl
[5] Adventure -
http://dh.crc.nd.edu/sandbox/cyl/catalog/details/adventurewithapa00ferriala.html
[6] Catholic Youth Literature - http://dh.crc.nd.edu/sandbox/cyl/catalog/
[7] ELIS - http://freeyourmetadata.org/named-entity-extraction/
--
Eric Lease Morgan
University of Notre Dame
574/631-8604
