On Sun, Aug 22, 2010 at 5:24 PM, Mitja Kleider <mi...@kleider.name> wrote: > Richard Fairhurst wrote: >> I think you're misreading how the relation integrity checks work. The > API >> only checks whether an entity is still visible when the entity is > _added_ >> to a relation. >> For performance reasons, it does not check the entities >> formerly in the relation. > > Thanks for clarifying. That is probably the cause. > > >> It assumes that they're ok, which in the case of historic >> (pre-integrity check) data may not be the case. See preconditions_ok? in >> app/models/relation.rb in the Rails port for the code. > > When were the integrity checks introduced?
many of them were in-place before API 0.6 went live on 2009-04-21, but 0.6 (i think) included far more of them and in a more consistent manner. > Are you sure that checking only _added_ entities is enough? I read it like > that: for performance reasons the checks are done when the entity is > deleted (way node in the next case). when a referenced entity is deleted it is checked for membership in any way or relation and, if it's a member of another entity, the deletion is disallowed. when a new entity is created, or elements added to an existing entity, then those members are checked to ensure they're not deleted. in a fully serialised model this should be enough to prevent any referential integrity problems. but... > Looking at the other example > ("http://www.openstreetmap.org/api/0.6/way/60263972/1"), the deleted nodes > contained in that way (e.g. > "http://www.openstreetmap.org/api/0.6/node/338392109/5") were deleted May > 2010 (2010-05-26). > > In my understanding it still should not have happened, or am I missing > something? unfortunately, we don't have a fully serialised model and many uploads, particularly from potlatch, execute in parallel. this leads to the case above where the node is being deleted and the way created at very similar times, but in different transactions. the timeline probably looks like: 1. (transaction A) request is received for node deletion. checks in software indicate that it isn't part of any ways or relations. the node is deleted. 2. (transaction B) request is received for way creation. checks in software indicate that the nodes are all not-deleted (since transaction A hasn't committed yet). the way is created. 3. (transaction B) commits without problems, as the FK constraint is satisfied. 4. (transaction A) commits without problems. since nodes are stored in the database with a "visible" flag to indicate whether they're deleted or not, the FK constraint can't catch this error. it's a similar story with relation members, as the current schema can't be FK checked. API 0.6 drastically improved things over 0.5 in its referential integrity, but we didn't manage to catch all the problems. hopefully we'll get them in 0.7. until then, when you find one of these it's best just to fix it. cheers, matt _______________________________________________ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev