you can download as much as you like for personal use (which includes
monging to extract the kind of metadata you're gathering)
carry on! we love it!
-t
I did consider adding stuff from other feeds, in which case I think
it would be limited to that specific area (or, say, 100 miles around
the area). The main problem with that idea is that it would mean
getting lots of whole articles from the BBC News website, which they
might not like. And I don't want to be in violation of the terms and
conditions ;).
Any clarification from BBC people on downloading articles from the
site like that would be useful.
The other problem with that idea is that the script takes ages to get
just the one feed and do stuff with it. Around 2 or 3 seconds per
news story.
If I can do that though, I would like to. I would also like to
optimise the script a bit.
Duncan
David Tattersall wrote:
Duncan,
You could see which feed the article is from (i.e. BBC North West)
and exclude any possible place names that lie out side of the bounds
of this area. You may get news items referring to other places, but
its unlikely if it was specifically filed under a certain feed.
David
From:
<mailto:[EMAIL PROTECTED]>[EMAIL PROTECTED]
[<mailto:[EMAIL PROTECTED]>mailto:[EMAIL PROTECTED]
On Behalf Of Duncan Barclay
Sent: 10 June 2005 7:15
To: <mailto:[email protected]>[email protected]
Subject: [backstage] News on a map
Hi,
There has been a lot of talk about adding geographical data to the
BBC RSS feeds recently, with many people saying that it isn't
possible to do it in a way that isn't manual. I agree that it isn't
100% accurate doing it automatically, but I have found that it is
possible to do it reasonably accurately.
Yes, that's right. I made another prototype :).
It can be found at
<http://backstage.min-data.co.uk/geotagged/>http://backstage.min-data.co.uk/geotagged/
. There are several problems with it at the moment, such as the fact
that it doesn't update ever as it takes too long to update for me to
be happy to make it update regularly (I will try to optimise the code
when I have the time).
So far I haven't seen it get a news story completely wrong, but I
will have to check it over a longer period of time than 2 days.
Some of you are probably wanting to know how it works out where a
news item is from, so I might as well say. Firstly it gets the news
article from the BBC News site (hope this is ok, don't want to be
breaking the T&Cs), strips it down to the actual article (not
replies, extra info, etc) and then works out which words could be
place names, quite simply by working out which ones have a capital
letter to start with. It then removes some common words and checks
the remaining ones in a database. At the same time it retrieves the
longitude and latitude of those locations.
Then, if there are no locations, it basically ignores the article as
there is no way to plot it on the map. That is why there are so few
articles there. If there is one location and it is mentioned enough
times in the text (number of times varies depending on the length of
the article), that point is plotted. If there are 2 or more, it
works out which two points are closest to each other, and if they are
less than 50 miles apart it plots the place that is mentioned first
in the article.
As I mentioned, there are a few problems with it. I am not entirely
clear what the rules are about scraping the BBC News site for use
purely by the script (the description and title for the articles all
come from the RSS feed), so some guidelines on that would be useful.
There is also a problem where it won't work with places with two
words in the name, which isn't ideal.
If any of you have any comments about it (and I am sure you will),
please tell me. All feedback on this would be appreciated.
Thanks,
Duncan
This e-mail, and any attachment, is confidential. If you have received
it in error, please delete it from your system, do not use or disclose
the information in any way, and notify me immediately. The contents of
this message may contain personal views which are not the views of the
BBC, unless specifically stated.
___
--
