Dominic, Thanks for the comments. Actually, this is also my field and I was just providing a very general outline of approaches. In reality, I would be more inclined to use some mixture of shallow parsing and some sort of semantic tagging based bootstrapping. My current interests, however, lie in the interpretation of these phrases and so I am keen to follow through on my experiment for using the US placename database as a core to a new approach. I hope to start posting some stats to this list regarding the database once I run a few experiments.
As for on/off topic. I think this forum is a great place for different interest to collide. Having worked in a number of inter-disciplinary contexts, there is so much that is not acheived due to lack of communication between researchers, etc. in different fields. By providing details or even high level descriptions of new areas here, other contributors can learn something and also bring a new perspective which is great for everyone! I'll have a look at your news stuff as soon as I am on a windows box... Matt Hurst Senior Research Scientist, Intelliseek, Inc BlogPulse: www.blogpulse.com On 5/28/05, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Hi Matthew, > > As this is pretty much my field, and at the risk of leading the list slightly > off-topic, I thought I'd add some comments. Hopefully they might be of > interest to others who are dealing with mapping too. > > > The problem of scanning a piece of text and locating references to named > > objects is often called entity recognition or extraction. There are other > > steps required which transform the piece of text into some semantic > > format. So the text > > 'I live in London' > > can be scanned for the string 'London' which is a mention of a place > > with a specific lat/lon. > > Yes, but 'I live in' is, of course, not always followed by a place name. > Checking the Bank of English (a 450 million word database of genuine examples > of world English, based at Birmingham University in England), we find the > phrase "I live in" occurs 984 times. The most common words after 'in' are > (in order): > > a, the, london, hope, an, and, new, north, my, this, is, fear, such, i, one, > leicester, it, what, england, west, east, south, britain > > To complicate matters still further, In 13 cases, 'London' is separated from > 'in' by another word: > I live in (North|South|East|Tooting,) London > Is 'North London' the same place as 'London' and, if it is, why is 'New York' > not the same place as 'York'? (ie. We have to rely on the computer realising > that 'North' is an adjective, as it normally is, but conversely 'New' is part > of a multi-word unit making up the Proper Noun, and not assuming its normal > role of adjective.) > > > Many attempts at this problem in general only go half way and locate > > mentions via patterns in text (e.g. the word 'in' followed by a capitalized > > word suggests that the capitalized word is a location of some sort). What > > we here are interested in is going all the way to the semantics (in this > > case the lat lat and probably the unique identifier mentioned later by > > Barry). > > Personally, I would worry about the idea of using capitalised words as a > basis for this. What we probably ought to be doing is using more complex > word patterns to find places. We'd want any process we came up with to work > in any language, including for example German, where all nouns, not just > proper nouns, are capitalised. In speech we can't capitalise letters, and so > it must be possible to uncover place names without relying on this, although > this causes interesting issues with cases like "I live in Hope." (attested in > the Bank of English), where 'Hope' is actually a place. (The computer would > have to read forward, finding the references to 'village' etc. to be able to > interpret 'Hope' as a place, just as we would in our minds if somebody said > this phrase to us.) > > To take this issue a step further, I found this example in the Bank of > English: > "That's a pretty good description of San Francisco, actually. In fact, I live > in what is probably one of the ultimate tourist destinations. Our beaches are > lousy and there are lots of drawbacks, but on some levels San Francisco > represents an unfulfilled dream..." > Here, the place is obviously "San Francisco", but the place name is well > separated from the node phrase "I live in". We really need to find ways to > get computers to analyse this just as we do mentally, so that we could > substitue "San Francisco" with anything, and have the computer realise it > must be a place name, whether or not it is in a list of known places. As you > suggest in talking about the tsunami, unknown places are just as important as > known ones! > > On an unrelated note entirely, the members of the list might not yet have > seen my prototype at http://www.domsmith.co.uk/bbcnews > The aim was to produce a window that you can leave on your desktop and view > news stories one by one. It has been implemented in XHTML+SMIL which sadly > means it is only Internet Explorer 6 compatible at the moment, but was never > really intended for anything more than to demonstrate what adding the SMIL > timing elements to XHTML can do. Enjoy! > > Best wishes, > > Dominic. > > -- > From: Dominic Smith > MPhil postgraduate student in Corpus Linguistics, > University of Birmingham. > > http://www.domsmith.co.uk > >
