On Thu, Oct 03, 2013 at 03:31:20PM +0200, César Martínez Izquierdo wrote:
> That sounds interesting. I am not familiar with Nominatim, but I have
> correctly understood, the result is a Postgres/postgis database with all
> those polygons and hierarchies. This could be an interesting approach as
> the post-processing could be directly done there using PostGIS predicates.
Yes, exactly.
> However, I am not sure about the generated hierarchies, as they don't keep
> all OSM admin_levels into account (at least in the nomenclature: country,
> state, county)
It keeps the levels internally and also uses the full levels for the
hierarchies. The levels are only grouped for output.
> and I see clear errors for Spain using the link [2] that you
> provided. It would be interesting to know how these hierarchies are
> generated (just OSM tags and geometric relations, external database, etc).
The hierarchies are built with OSM data only using the tagged admin_levels and
relating the polygon geometries. Basically, the parent is defined as the
area that covers the object that has the next smaller admin_level. It
gets a bit more complicated when place nodes are involved.
Sarah
> 2013/10/3 Sarah Hoffmann <lon...@denofr.de>
>
> > On Thu, Oct 03, 2013 at 12:32:36PM +0200, César Martínez Izquierdo wrote:
> > > Hi Frederik, regarding software, I am already familiar with Mapit scripts
> > > code, which are able to extract admin boundary polygons for each level
> > (it
> > > is not creating relationships though). How do you see Nominatim or
> > > Osmium/osmjs better for the purpose?
> >
> > Nominatim already does a lot of the stuff you are planning to do. It
> > creates geometries for admin boundaries from OSM data and puts them
> > in a proper hierarchy. It is able to process updates and in the
> > process makes sure that boundaries do not just disappear if somebody
> > breaks the relation. If you only process the data that interests you
> > (boundary=administrative and place nodes) it is not even that
> > resource-hungry.
> >
> > It does have support for listing broken boundaries [1] and during the
> > last hack day Brian has written a proof-of-concept for browsing admin
> > hierarchies[2]. There is even a script to dump objects within a certain
> > boundary[3] which could be easily extended to dump entire hierarchies.
> > All these functions should currently be used with care though. There are
> > known bugs and the output needs to be improved to make them really
> > usable.
> >
> > Basically, most of the work to do would be on the output side, the
> > heavy lifting of processing the data is already done. Well, apart
> > from checking the OSM boundaries against reality. But I think wiki data
> > is a good starting point for that.
> >
> > > I will probably try the first option, but I expect that any of them would
> > > be costly to maintain if there are frequent changes of the tagging scheme
> > > for each country. (But nobody said it would be an easy task :-)
> >
> > Making boundary tagging more visible will hopefully help to stablize
> > and unify the tagging schemas. The less country-specific exceptions
> > the better.
> >
> > Sarah
> >
> > [1] http://nominatim.osm.org/polygons
> > [2] http://nominatim.openstreetmap.org/hierarchy.php?place_id=97944985
> > (This is for demonstration only, please do not scrape. It will not
> > always give you the expected results anyways because there
> > is a
> > known bug with parenting which still lingers in the DB.)
> > [3]
> > https://github.com/lonvia/Nominatim/blob/export-script/utils/export.php
> >
> > > 2013/10/2 Frederik Ramm <frede...@remote.org>
> > >
> > > > Hi,
> > > >
> > > > On 02.10.2013 18:23, César Martínez Izquierdo wrote:
> > > > > I plan to create and make easily available a world-wide
> > administrative
> > > > > layer based on OSM data, ideally including existing administrative
> > codes
> > > > > (ISO, NUTS in Europe, etc) for each level and producing regular
> > updates
> > > > > (for instance once a year).
> > > >
> > > > This is something I have been thinking about for a long time (but never
> > > > written any usable code).
> > > >
> > > > Nominatim is probably a good starting point - a better one than MapIt I
> > > > should think.
> > > >
> > > > If you're only after extracting certain relation polygons then you
> > could
> > > > as well use osmjs (part of Osmium) and have it generate shape files for
> > > > you, or adapt the shapefile/ogr export samples in Osmium; this will not
> > > > yet give you a hierarchy, only individual boundaries, and you have to
> > > > find out the hierarchy yourself.
> > > >
> > > > Finding out the hierarchy is going to be tricky. Nominatim does go to
> > > > some lengths to do that already. It sounds easy ("find all polygons
> > with
> > > > an admin level smaller than X where this polygon I'm looking at lies
> > > > in"). But in reality you will encounter at least:
> > > >
> > > > * missing polygons on all levels - sometimes simply not mapped,
> > > > sometimes missing by design, e.g. Germany has some areas where admin
> > > > levels 8, 6, and 4 coincide, these are mapped as admin_level 4, so draw
> > > > a map of all admin_level 8 areas in Germany and you have lots of holes
> > > > in them
> > > >
> > > > * broken polygons on all levels; brokenness changes by the day, i.e.
> > > > what is working today may be broken tomorrow and vice versa
> > > >
> > > > * occasionally (e.g. Japan) linear regional boundaries that simply go
> > > > from coast to coast without including the coastline
> > > >
> > > > * occasionally (e.g. Chile) a regional boundary that is not a
> > > > multipolygon relation but instead a grouping of smaller regional
> > entities
> > > >
> > > > * sometimes small geometric inaccuracies mean that e.g. a state
> > boundary
> > > > fails the "is-in" test for the country boundary because there's just a
> > > > little square metre somewhere that is mapped as belonging to the state
> > > > but not the country
> > > >
> > > > * overlapping admin polygons of the same admin level
> > > >
> > > > I think that ate the very least you need to run the evaluation
> > regularly
> > > > and compare: Do I have new polygons this week - have others vanished,
> > > > and if so, is that because they were explicitly deleted/replaced, or
> > > > were they just accidentally broken and I should continue to use last
> > > > week's?
> > > >
> > > > What we would really need though, is something much bigger: A separate
> > > > database of admin hierarchies, where people could - in a crowdsourced
> > > > manner - record things like:
> > > >
> > > > "There is an adminlevel 2 entities called Germany"
> > > > "It is divided into 16 exclusive adminlevel 4 entities with the
> > > > following names: ..."
> > > > "These 16 entities cover the area of Germany completely (no holes or
> > sea
> > > > areas that would be outside of one of the entities)"
> > > > "The adminlevel 4 entity named 'Brandenburg' is divided in X adminlevel
> > > > 6 entities..."
> > > >
> > > > and so on. A tree of arbitrary size where people can add and edit at
> > will.
> > > >
> > > > Now you will say "but this tree could be generated from OpenStreetMap",
> > > > and I grant that one could attempt to build such a tree but it will
> > > > always be faulty and reflect the current brokenness of geometries in
> > > > OSM. One could *start* with an OSM-generated tree, but after that, the
> > > > tree must be kept separate. People should be able to add stuff to the
> > > > tree even when it is not in OpenStreetMap - "there should be an
> > > > adminlevel 8 boundary called so-and-so". A regularly-running process
> > > > would then compare the tree to OpenStreetMap, and generate error
> > reports
> > > > that can be presented visually:
> > > >
> > > > "The tree says that there should be a region called X in Germany but
> > OSM
> > > > doesn't have one."
> > > >
> > > > "There is an area here that is not covered by any adminlevel 4 area but
> > > > the tree says that taken together the adminlevel 4 areas must cover all
> > > > of the country."
> > > >
> > > > "The tree claims there should be a region called X but in OSM there's
> > > > only a region with the similar name Y, which one is correct?"
> > > >
> > > > and so on. - I would expect the tree to be much more stable than the
> > > > data in OSM. Most of all, the tree could be worked on independently,
> > > > even by people unfamiliar with OSM. Of course the tree could link to
> > OSM
> > > > objects but these links would regularly be checked and perhaps even
> > > > changed by the automated comparison system.
> > > >
> > > > As I said, a simple export is easy but it will have too many weaknesses
> > > > - you couldn't even say to what level a country is "complete". Some
> > > > people have started to link regional boundaries in OSM together with
> > > > concepts like "subarea" but I don't think this can replace an external
> > > > country structure tree because the tree could describe what is expected
> > > > to be there whereas in OSM you never know if some thing is missing
> > > > because it doesn't exist (cf. my example about the "missing" admin6/8
> > > > boundaries in Germany) or if it just hasn't been mapped yet, or is
> > broken.
> > > >
> > > > It would be great to see someone drive this important topic but it
> > > > certainly isn't something you can set up in a week or two ;)
> > > >
> > > > Bye
> > > > Frederik
> > > >
> > > > --
> > > > Frederik Ramm ## eMail frede...@remote.org ## N49°00'09"
> > E008°23'33"
> > > >
> > > > _______________________________________________
> > > > talk mailing list
> > > > talk@openstreetmap.org
> > > > https://lists.openstreetmap.org/listinfo/talk
> > > >
> > >
> > >
> > >
> > > --
> > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> > > César Martínez Izquierdo
> > > GIS developer
> > > - - - - - - - - - - - - - - - - - - - -
> > > ETC-SIA: http://sia.eionet.europa.eu/
> > > Universitat Autònoma de Barcelona (SPAIN)
> > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> >
> > > _______________________________________________
> > > talk mailing list
> > > talk@openstreetmap.org
> > > https://lists.openstreetmap.org/listinfo/talk
> >
> >
>
>
> --
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> César Martínez Izquierdo
> GIS developer
> - - - - - - - - - - - - - - - - - - - -
> ETC-SIA: http://sia.eionet.europa.eu/
> Universitat Autònoma de Barcelona (SPAIN)
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
_______________________________________________
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
