Re: [OSM-talk] Searching for recently added names not working

[David Earl]

On 25/08/2008 00:27, Brett Henderson wrote:
> On Thu, Aug 21, 2008 at 8:19 PM, David Earl <[EMAIL PROTECTED] 
> > wrote:
> Sorry about this. The rules changed under my feet!
> 
> Are you sure?  I didn't remember changing anything in this space so 
> checked svn.  My original osmosis code import to svn in September 2007 
> had the xml tag named as "create".  The same code is used for creating 
> the xml within the bz2 and gz files so there shouldn't be any difference 
> between the two.  The difference between the bz2 and gz processes was in 
> the extract automation, not the database and xml code.
> 
> I wouldn't bring it up except that it sounds like you might be missing 
> more data than just that which changed in the past couple of weeks.

Hmm, I think you are right. I can't see when this did start happening 
though.

I believe it was correct when the incremental updates to namefinder 
started in March (20 March to be precise; I believe the daily diffs 
contained 'add' at that point - is that your understanding too?).

I can find unchanged entries from late May that are in the index; OTOH 
the problem may be more subtle in that if the last element before 
'create' was 'modify', then the 'create's may be interpreted as modify, 
but if it was delete, then as delete, so sometimes entries might have 
been, added sometimes not.

Could it have been the change you mailed about on June 24? Or were the 
bzip files unchanged at that point, and only the new gz ones (which I 
only switched to on August 8) affected?

Am I imagining it, or was there a point at which 'create' replaced 'add' 
in the daily files?

I suspect I am going to have to reload from a full planet, in which case 
I won't do anything urgently as it will be useful to add a few more 
categories of object to the index and make a few changes to the 
proximity algorithm before I do.

David


___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Searching for recently added names not working

[Brett Henderson]

On Thu, Aug 21, 2008 at 8:19 PM, David Earl <[EMAIL PROTECTED]>wrote:
...

>
> The problem was that when I switched it to using the daily planet gz
> instead of bz2 files, I hadn't realized that the name of the XML tag for
> additions also changed from " all look reasonable: it was showing a large number of ways etc being
> processed, but of course it was only counting modified ones, not new
> ones, skipping the entries enclosed by "create" completely. So I only
> discovered this when I searched for some streets I added this week and
> couldn't find some but did find others (those I had later modified) and
> investigated why.

...

>
>
> Sorry about this. The rules changed under my feet!
>
> Are you sure?  I didn't remember changing anything in this space so checked
svn.  My original osmosis code import to svn in September 2007 had the xml
tag named as "create".  The same code is used for creating the xml within
the bz2 and gz files so there shouldn't be any difference between the two.
The difference between the bz2 and gz processes was in the extract
automation, not the database and xml code.

I wouldn't bring it up except that it sounds like you might be missing more
data than just that which changed in the past couple of weeks.

Brett
___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Searching for recently added names not working

[David Earl]

All up to date now.

David

On 21/08/2008 11:19, David Earl wrote:
> I'm afraid the name finder index has got a problem which means that 
> index entries for new objects (node/way/relation) added since August 8 
> are missing. (Actually it is a bit more subtle than that, depending on 
> whether the object was also later modified, and whether there happened 
> to be no deletions of objects of the same type on any particular day).
> 
> The problem was that when I switched it to using the daily planet gz 
> instead of bz2 files, I hadn't realized that the name of the XML tag for 
> additions also changed from " all look reasonable: it was showing a large number of ways etc being 
> processed, but of course it was only counting modified ones, not new 
> ones, skipping the entries enclosed by "create" completely. So I only 
> discovered this when I searched for some streets I added this week and 
> couldn't find some but did find others (those I had later modified) and 
> investigated why.
> 
> Fortunately all the daily diffs back to 8/8 are still there, so I am now 
> going to suspend the daily update, re-run all the updates for the last 
> two weeks, and then catch up with tonight's and maybe tomorrow's later - 
> the longer update may well take more than a day to run. In the meantime, 
> searching will be very slow and may not find entries which are removed 
> during the file processing phase but have not yet been processed in the 
> update phase (which will be many hours apart).
> 
> Sorry about this. The rules changed under my feet!
> 
> David


___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk

[OSM-talk] Searching for recently added names not working

[David Earl]

I'm afraid the name finder index has got a problem which means that 
index entries for new objects (node/way/relation) added since August 8 
are missing. (Actually it is a bit more subtle than that, depending on 
whether the object was also later modified, and whether there happened 
to be no deletions of objects of the same type on any particular day).

The problem was that when I switched it to using the daily planet gz 
instead of bz2 files, I hadn't realized that the name of the XML tag for 
additions also changed from "http://lists.openstreetmap.org/listinfo/talk