Re: [OSM-talk] Administrative boundaries export

2013-10-04 Thread César Martínez Izquierdo
I have finally found the way. I leave some examples here:

As JSON:
http://www.wikidata.org/w/api.php?action=wbgetentitiesids=Q5720languages=enformat=json
As XML:
http://www.wikidata.org/w/api.php?action=wbgetentitiesids=Q5720languages=enformat=xml

Using title:
http://www.wikidata.org/w/api.php?action=wbgetentitiessites=enwikititles=Valencian%20Communitylanguages=enformat=xml

Regards,

César


2013/10/4 Eugene Alvin Villar sea...@gmail.com

 On Fri, Oct 4, 2013 at 1:35 AM, César Martínez Izquierdo 
 cesar@gmail.com wrote:

 Thanks Eugene, that looks really promising.
 I've seen there is an API to query Wikidata (results can be list of
 Wikidata item IDs encoded as JSON), but I don't see the way to get the item
 itself as JSON (or any other parseable format). Is it on the way?


 Unfortunately, I am not up-to-par with the API side of Wikidata. I assume
 that every bit of data on Wikidata can or will be accessible through APIs.
 Otherwise, it would limit the usefulness of Wikidata if we resort to
 scraping the HTML page.





-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-SIA: http://sia.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread Martin Koppenhoefer


 Am 02/ott/2013 um 23:52 schrieb Christian Quest cqu...@openstreetmap.fr:
 
 UK level 4 is on the maritime borders (island culture ?) where most other 
 European countries stop on the coastline... tagging bio-diversity is not 
 helpful !


well, maybe this is how things are correct? years ago I stumbled upon Liberia 
having a 200NM maritime border. Checking with other sources it turned out that 
they indeed claim this area. Looking now at the osm map someone has 
normalized the coastline leaving them the same maritime border than the rest, 
and it looks better but I'm not sure it also is more correct.

The world is divers and the map should represent how things are, even if this 
seems more chaotic than nicely structured.

cheers,
Martin


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread Christian Quest
Well, I think I fully understand all the diversity, but when you want the
shape of UK or Liberia, I presume most people expect the land, not the
maritime boundaries claimed ones or whatever.

This does not prevent to have 2 boundary relations, one for land boundary
and one including maritime ones (that's the case for France for example +
one including overseas land), but make sure the tagging is homogeneous
worldwide as far as possible to allow easy reuse of data.

I've been asked too about all maritime boundaries of the world... I tried
to extract that but it was too inconsistent to be useable.



2013/10/3 Martin Koppenhoefer dieterdre...@gmail.com



  Am 02/ott/2013 um 23:52 schrieb Christian Quest cqu...@openstreetmap.fr
 :
 
  UK level 4 is on the maritime borders (island culture ?) where most
 other European countries stop on the coastline... tagging bio-diversity is
 not helpful !


 well, maybe this is how things are correct? years ago I stumbled upon
 Liberia having a 200NM maritime border. Checking with other sources it
 turned out that they indeed claim this area. Looking now at the osm map
 someone has normalized the coastline leaving them the same maritime
 border than the rest, and it looks better but I'm not sure it also is
 more correct.

 The world is divers and the map should represent how things are, even if
 this seems more chaotic than nicely structured.

 cheers,
 Martin




-- 
Christian Quest - OpenStreetMap France
Un nouveau serveur pour OSM... http://donate.osm.org/server2013/
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread César Martínez Izquierdo
Thanks for all the advises and suggestion. I agree that in order to achieve
this, at least 2 tables/trees/dbs have to be maintained:

1 - The relationship of each admin_boundary on a certain level with its
parent (and the opposite) and whether this same boundaries applies for
other admin_boundaries (as suggested for some areas of Germany)
2 - For each country, how to distinguish the land mass from territorial
waters. I am more interested on mapping the land mass, but the territorial
waters could be also generated if we have this distinction.

Regarding 1, I think it would be reasonable to try to codify it within OSM
(for instance using is_in tag), but I think the tagging scheme should be
further developed to correctly capture all the information we need, so I
will generate an external table/database instead. Probably, a table per
admin_level containing relation-id, ISO 3166 code would be enough (but I
have to study what should be used for deeper subdivision levels), as it
will contain the father-children relationships and also would solve the
problem with a boundary being the same for different admin_levels. The
algorithm could check against OSM data for code and for relation-id when
missing, and report any added or removed boundaries compared with previous
version.

Regarding 2, for the long term I thing separate relationships should be
present in OSM for land mass and territorial waters (this also involves
clarifying the tagging scheme), but for the moment I will try to manage
with existing tags.

I assume it will not be a small task, so I will try to be pragmatic and
start exporting what can be exported using a straightforward way and then
refining the approach little by little. Probably, in order to refine these
approaches it would be helpful to have some viewer similar to the one on
openstreetmap.fr (to easily see what should be corrected on OSM data or on
the extraction algorithms), but for the moment this is outside of my plans.

I expect results on the scale of months/years rather than days/weeks
(according to my planned dedication), but I will keep you informed when I
get some news.

Eugene, I am also interested on your proposal to store on Wikidata a
table/database similar to the one described on 1, so any further details on
available infrastructure, technologies in use, work already done, etc are
welcome.

Cheers,

César Martínez Izquierdo



2013/10/3 Eugene Alvin Villar sea...@gmail.com

 On Thu, Oct 3, 2013 at 6:52 AM, Christian Quest 
 cqu...@openstreetmap.frwrote:

 UK level 4 is on the maritime borders (island culture ?) where most other
 European countries stop on the coastline... tagging bio-diversity is not
 helpful !


 This is actually another point to consider when extracting admin
 boundaries from OSM data.

 My personal view is that the admin boundary marks the boundary where the
 administrative entity exercises jurisdiction. Under UNCLOS, nations are
 allowed to exercise full sovereignty over internal waters (which includes
 water seaward of the coastline but within the baselines) and almost full
 sovereignty (foreign ships are allowed innocent passage) for territorial
 waters (up to 12 nautical miles from the baselines). So I think that using
 the maritime boundaries as the admin_level=2 boundary is not incorrect and
 this is reflected in OSM.

 Use of maritime boundaries for admin_level=3 and higher (such as with UK)
 depends on how the particular nation interprets its internal
 maritime/fisheries laws. In my country, we have municipal waters which
 can be up to 15 kilometers from the shoreline and that's where
 municipalities can exercise jurisdiction.

 ___
 talk mailing list
 talk@openstreetmap.org
 https://lists.openstreetmap.org/listinfo/talk




-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-SIA: http://sia.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread Martin Koppenhoefer
2013/10/3 César Martínez Izquierdo cesar@gmail.com

 2 - For each country, how to distinguish the land mass from territorial
 waters. I am more interested on mapping the land mass, but the territorial
 waters could be also generated if we have this distinction.



no, the landmass is the land inside the territorial waters, but the
territorial waters are not the extension of the coastline by 12seamiles but
the extension of the baseline. We generally do not have the baselines in
OSM as far as I am aware of.

cheers,
Martin
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread Martin Koppenhoefer
2013/10/3 Christian Quest cqu...@openstreetmap.fr

 Well, I think I fully understand all the diversity, but when you want the
 shape of UK or Liberia, I presume most people expect the land, not the
 maritime boundaries claimed ones or whatever.



yes, but you don't need a special relation for this, you only have to find
out the coastlines inside the territory (and where they cross the national
borders so you can find out the land borders to other countries).




 This does not prevent to have 2 boundary relations, one for land boundary
 and one including maritime ones



you don't need them, and you will put unneccessary heavy load on the db
creating a relation with all coastlines and the land side borders in it (as
the coastlines are much more detailed than the baseline). In the end,
that's the reason we do coastlines with shapefiles and not with huge
polygons. We used to have a landmass relation for Italy as well, because
some well meaning mappers have created them for us, but it was continuously
broken and augmented the complexity for coastline edits with an
unneccessary complication. It grew in short time to 928 versions, so after
a discussion on talk-it, and with nobody needing it, we decided to delete
it some weeks ago: http://www.openstreetmap.org/browse/relation/957374



 (that's the case for France for example + one including overseas land),
 but make sure the tagging is homogeneous worldwide as far as possible to
 allow easy reuse of data.



My suggestion would be to delete the French landmass relation as well for
homogenisation ;-)

cheers,
Martin
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread César Martínez Izquierdo
Hi Frederik, regarding software, I am already familiar with Mapit scripts
code, which are able to extract admin boundary polygons for each level (it
is not creating relationships though). How do you see Nominatim or
Osmium/osmjs better for the purpose? Reading osmjs documentation, I see it
could be used to build a similar script to mapit one (which uses a local
instance of OSM3S api). Do you think it would be faster?

I envisage 2 possible approaches:
- using one of these tools to do a first export, which should be later
processed by a separate script in order to make sense on the
particularities of each country.
- creating an export script from scratch (similar to or based on Mapit
scripts) that select the right relations and tags for each country. It
should be faster but probably more costly to implement

I will probably try the first option, but I expect that any of them would
be costly to maintain if there are frequent changes of the tagging scheme
for each country. (But nobody said it would be an easy task :-)

Cheers,

César

2013/10/2 Frederik Ramm frede...@remote.org

 Hi,

 On 02.10.2013 18:23, César Martínez Izquierdo wrote:
  I plan to create and make easily available a world-wide administrative
  layer based on OSM data, ideally including existing administrative codes
  (ISO, NUTS in Europe, etc) for each level and producing regular updates
  (for instance once a year).

 This is something I have been thinking about for a long time (but never
 written any usable code).

 Nominatim is probably a good starting point - a better one than MapIt I
 should think.

 If you're only after extracting certain relation polygons then you could
 as well use osmjs (part of Osmium) and have it generate shape files for
 you, or adapt the shapefile/ogr export samples in Osmium; this will not
 yet give you a hierarchy, only individual boundaries, and you have to
 find out the hierarchy yourself.

 Finding out the hierarchy is going to be tricky. Nominatim does go to
 some lengths to do that already. It sounds easy (find all polygons with
 an admin level smaller than X where this polygon I'm looking at lies
 in). But in reality you will encounter at least:

 * missing polygons on all levels - sometimes simply not mapped,
 sometimes missing by design, e.g. Germany has some areas where admin
 levels 8, 6, and 4 coincide, these are mapped as admin_level 4, so draw
 a map of all admin_level 8 areas in Germany and you have lots of holes
 in them

 * broken polygons on all levels; brokenness changes by the day, i.e.
 what is working today may be broken tomorrow and vice versa

 * occasionally (e.g. Japan) linear regional boundaries that simply go
 from coast to coast without including the coastline

 * occasionally (e.g. Chile) a regional boundary that is not a
 multipolygon relation but instead a grouping of smaller regional entities

 * sometimes small geometric inaccuracies mean that e.g. a state boundary
 fails the is-in test for the country boundary because there's just a
 little square metre somewhere that is mapped as belonging to the state
 but not the country

 * overlapping admin polygons of the same admin level

 I think that ate the very least you need to run the evaluation regularly
 and compare: Do I have new polygons this week - have others vanished,
 and if so, is that because they were explicitly deleted/replaced, or
 were they just accidentally broken and I should continue to use last
 week's?

 What we would really need though, is something much bigger: A separate
 database of admin hierarchies, where people could - in a crowdsourced
 manner - record things like:

 There is an adminlevel 2 entities called Germany
 It is divided into 16 exclusive adminlevel 4 entities with the
 following names: ...
 These 16 entities cover the area of Germany completely (no holes or sea
 areas that would be outside of one of the entities)
 The adminlevel 4 entity named 'Brandenburg' is divided in X adminlevel
 6 entities...

 and so on. A tree of arbitrary size where people can add and edit at will.

 Now you will say but this tree could be generated from OpenStreetMap,
 and I grant that one could attempt to build such a tree but it will
 always be faulty and reflect the current brokenness of geometries in
 OSM. One could *start* with an OSM-generated tree, but after that, the
 tree must be kept separate. People should be able to add stuff to the
 tree even when it is not in OpenStreetMap - there should be an
 adminlevel 8 boundary called so-and-so. A regularly-running process
 would then compare the tree to OpenStreetMap, and generate error reports
 that can be presented visually:

 The tree says that there should be a region called X in Germany but OSM
 doesn't have one.

 There is an area here that is not covered by any adminlevel 4 area but
 the tree says that taken together the adminlevel 4 areas must cover all
 of the country.

 The tree claims there should be a region called X but in OSM there's
 only a region 

Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread Frederik Ramm
Hi,

On 10/03/13 12:32, César Martínez Izquierdo wrote:
 Hi Frederik, regarding software, I am already familiar with Mapit
 scripts code, which are able to extract admin boundary polygons for each
 level (it is not creating relationships though). How do you see
 Nominatim or Osmium/osmjs better for the purpose? Reading osmjs
 documentation, I see it could be used to build a similar script to mapit
 one (which uses a local instance of OSM3S api). Do you think it would be
 faster?

I would think so. The Mapit stack is very convoluted - as far as I
remember, it first extracts stuff from a planet file with Osmosis, then
imports that into osm3s (aka overpass), then makes queries from there.
This is a very complicated way to do it - with an osmjs/osmium based
program you'd process the planet once and that's it (if using OGR output
like in the osmium_toogr example you can even generate KML directly),
and the time spend would very likely be much less than even the initial
Osmosis step of the Mapit stack. I'd assume it would take a couple hours
for a full export.

Of course if you're already familiar with the Mapit stack and it works
for you, that's fine too.

 - using one of these tools to do a first export, which should be later
 processed by a separate script in order to make sense on the
 particularities of each country.

Yes, I think that makes sense, however keep in mind that depending on
how you do the export, some things might already be broken at that
stage, and no chance for a program down the line to fix it. The export
process would probably have to be modified to export a collection of
broken things in addition to the working things, so that a script down
the line can then do something with the broken things.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09 E008°23'33

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread Sarah Hoffmann
On Thu, Oct 03, 2013 at 12:32:36PM +0200, César Martínez Izquierdo wrote:
 Hi Frederik, regarding software, I am already familiar with Mapit scripts
 code, which are able to extract admin boundary polygons for each level (it
 is not creating relationships though). How do you see Nominatim or
 Osmium/osmjs better for the purpose? 

Nominatim already does a lot of the stuff you are planning to do. It 
creates geometries for admin boundaries from OSM data and puts them 
in a proper hierarchy. It is able to process updates and in the 
process makes sure that boundaries do not just disappear if somebody
breaks the relation. If you only process the data that interests you 
(boundary=administrative and place nodes) it is not even that 
resource-hungry.

It does have support for listing broken boundaries [1] and during the
last hack day Brian has written a proof-of-concept for browsing admin
hierarchies[2]. There is even a script to dump objects within a certain 
boundary[3] which could be easily extended to dump entire hierarchies.
All these functions should currently be used with care though. There are
known bugs and the output needs to be improved to make them really
usable.

Basically, most of the work to do would be on the output side, the 
heavy lifting of processing the data is already done. Well, apart 
from checking the OSM boundaries against reality. But I think wiki data 
is a good starting point for that.

 I will probably try the first option, but I expect that any of them would
 be costly to maintain if there are frequent changes of the tagging scheme
 for each country. (But nobody said it would be an easy task :-)

Making boundary tagging more visible will hopefully help to stablize
and unify the tagging schemas. The less country-specific exceptions
the better.

Sarah

[1] http://nominatim.osm.org/polygons
[2] http://nominatim.openstreetmap.org/hierarchy.php?place_id=97944985
(This is for demonstration only, please do not scrape. It will not
always give you the expected results anyways because there is a
known bug with parenting which still lingers in the DB.)
[3] https://github.com/lonvia/Nominatim/blob/export-script/utils/export.php

 2013/10/2 Frederik Ramm frede...@remote.org
 
  Hi,
 
  On 02.10.2013 18:23, César Martínez Izquierdo wrote:
   I plan to create and make easily available a world-wide administrative
   layer based on OSM data, ideally including existing administrative codes
   (ISO, NUTS in Europe, etc) for each level and producing regular updates
   (for instance once a year).
 
  This is something I have been thinking about for a long time (but never
  written any usable code).
 
  Nominatim is probably a good starting point - a better one than MapIt I
  should think.
 
  If you're only after extracting certain relation polygons then you could
  as well use osmjs (part of Osmium) and have it generate shape files for
  you, or adapt the shapefile/ogr export samples in Osmium; this will not
  yet give you a hierarchy, only individual boundaries, and you have to
  find out the hierarchy yourself.
 
  Finding out the hierarchy is going to be tricky. Nominatim does go to
  some lengths to do that already. It sounds easy (find all polygons with
  an admin level smaller than X where this polygon I'm looking at lies
  in). But in reality you will encounter at least:
 
  * missing polygons on all levels - sometimes simply not mapped,
  sometimes missing by design, e.g. Germany has some areas where admin
  levels 8, 6, and 4 coincide, these are mapped as admin_level 4, so draw
  a map of all admin_level 8 areas in Germany and you have lots of holes
  in them
 
  * broken polygons on all levels; brokenness changes by the day, i.e.
  what is working today may be broken tomorrow and vice versa
 
  * occasionally (e.g. Japan) linear regional boundaries that simply go
  from coast to coast without including the coastline
 
  * occasionally (e.g. Chile) a regional boundary that is not a
  multipolygon relation but instead a grouping of smaller regional entities
 
  * sometimes small geometric inaccuracies mean that e.g. a state boundary
  fails the is-in test for the country boundary because there's just a
  little square metre somewhere that is mapped as belonging to the state
  but not the country
 
  * overlapping admin polygons of the same admin level
 
  I think that ate the very least you need to run the evaluation regularly
  and compare: Do I have new polygons this week - have others vanished,
  and if so, is that because they were explicitly deleted/replaced, or
  were they just accidentally broken and I should continue to use last
  week's?
 
  What we would really need though, is something much bigger: A separate
  database of admin hierarchies, where people could - in a crowdsourced
  manner - record things like:
 
  There is an adminlevel 2 entities called Germany
  It is divided into 16 exclusive adminlevel 4 entities with the
  following 

Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread Paul Norman
From: Martin Koppenhoefer [mailto:dieterdre...@gmail.com] 
Sent: Thursday, October 03, 2013 3:20 AM
Subject: Re: [OSM-talk] Administrative boundaries export

 you don't need them, and you will put unneccessary heavy load on the db 
 creating a relation with all coastlines and the land side borders in it 
 (as the coastlines are much more detailed than the baseline). 

Expanding on this, there are some large boundary relations which have 
all the coastline for an admin_level region and have over 10 000 
members. These are frankly harmful and impossible to work with. 

When I redid the Alaska boundaries, I put them at 3 miles out I believe. 

Frankly, I find the idea of using the coastline as an admin boundary 
rather silly. This would mean that if you step out 1 foot into the 
water, you've left the state or country.


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread Martin Koppenhoefer
2013/10/3 Paul Norman penor...@mac.com


 Frankly, I find the idea of using the coastline as an admin boundary
 rather silly. This would mean that if you step out 1 foot into the
 water, you've left the state or country.



indeed it seems to be different: en.wikipedia.org/wiki/Territorial_waters
look at the internal waters in the diagram, those allow to reduce the
admin-ways a lot by simplifying and not using the coastline.

cheers,
Martin
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread Eugene Alvin Villar
On Thu, Oct 3, 2013 at 9:18 PM, Martin Koppenhoefer
dieterdre...@gmail.comwrote:


 2013/10/3 Paul Norman penor...@mac.com


 Frankly, I find the idea of using the coastline as an admin boundary
 rather silly. This would mean that if you step out 1 foot into the
 water, you've left the state or country.



 indeed it seems to be different: en.wikipedia.org/wiki/Territorial_waters
 look at the internal waters in the diagram, those allow to reduce the
 admin-ways a lot by simplifying and not using the coastline.


Not quite. By default the baselines use the mean low-water line, basically
the coastlines, but not the coastlines in the OSM sense (high-water line).
Straight baselines *may* be used only in cases where the coastline is
deeply indented such as bays, coves, fjords, and the like, or if the state
claims to be an archipelagic state (whose baselines are all straight). So
for the more-or-less concave coastlines, the baselines would still be
complex. See: https://en.wikipedia.org/wiki/Baseline_%28sea%29
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread Eugene Alvin Villar
On Thu, Oct 3, 2013 at 6:35 PM, César Martínez Izquierdo 
cesar@gmail.com wrote:

 Eugene, I am also interested on your proposal to store on Wikidata a
 table/database similar to the one described on 1, so any further details on
 available infrastructure, technologies in use, work already done, etc are
 welcome.


Hi César, you can look at this Wikidata page for the German state of
Baden-Württemberg as an example: https://www.wikidata.org/wiki/Q985

It currently contains interesting properties and relationships that may be
what we need. Some interesting properties/relations are (especially the OSM
one):

country=Germany
capital=Stuttgart
type of administrative division=state of Germany
ISO 3166-2=DE-BW
GND identifier=4004176-1
contains administrative division=Regierungsbezirk Freiburg,
Regierungsbezirk Karlsruhe, Regierungsbezirk Stuttgart, Regierungsbezirk
Tübingen
is in the administrative unit=Germany
OpenStreetMap Relation ID=62611
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread Pieren
On Thu, Oct 3, 2013 at 2:16 PM, Paul Norman penor...@mac.com wrote:

 Frankly, I find the idea of using the coastline as an admin boundary
 rather silly. This would mean that if you step out 1 foot into the
 water, you've left the state or country.

All the states or countries maps I've seen in my life used the
coastline. It does not mean that the sovereignty stops at the water
line. It's just a convention.

Pieren

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread Martin Koppenhoefer
2013/10/3 Pieren pier...@gmail.com

 All the states or countries maps I've seen in my life used the
 coastline. It does not mean that the sovereignty stops at the water
 line. It's just a convention.



how many states or country maps have you seen in scale 1:1000?

cheers,
Martin
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread Martin Koppenhoefer
2013/10/3 Martin Koppenhoefer dieterdre...@gmail.com

 well, maybe this is how things are correct? years ago I stumbled upon
 Liberia having a 200NM maritime border.



in reference to this I have found a document today stating that the
president of Liberia has released an executive order on Jan 10th 2013 which
seems to reduce their respective claims to the standard:
http://emansion.gov.lr/doc/Executive_Order_No._48.pdf

cheers,
Martin
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread Maarten Deen

On 2013-10-03 14:53, Martin Koppenhoefer wrote:

2013/10/3 Pieren pier...@gmail.com

All the states or countries maps I've seen in my life used the
coastline. It does not mean that the sovereignty stops at the water
line. It's just a convention.

how many states or country maps have you seen in scale 1:1000?


All maps I have seen make a clear distinction between land and sea. Only 
a very small number of maps I've seen actually show borders on 
international waters.


So it all depends on what you want to show and how you visualise it. If 
you make a map of Germany or the Netherlands with one color within the 
borders in the sense of territorial waters, I think few people would 
recognise them.
For the netherlands, you would have a smooth coastline without islands 
(Zeeland and Waddeneilanden).


Regards,
Maarten

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread César Martínez Izquierdo
The important issue here is that there is lots of potential uses here
(layers are not used exclusively for creating maps, they can also be used
for analysis) and there are people wishing to use these datasets: the
administrative boundaries based on coastline (call them whatever you want),
the ones based on the territorial waters, and possible others.

Because of this reason, it would be useful to have them explicitly mapped
on OSM. It is not the same to just extract a relation and descendants and
build some polygons from there, than to extract the coastline and the
borders that are not coastline and try to see how they fit within their
containing country. It is feasible, but not practical.

However, as you suggested, it is important to keep an eye on
maintainability of OSM data when creating these relations. A simple
approach would be to create land mass relations having other relations as
members. For instance, in Spain such relationship could be easily created
from the admin_level=4 relations, containing just 19 members (as subareas).
As far as the admin_level=4 relations are correct, the superrelation should
be correct (if not, they should be corrected anyway). I don't know if such
approach would be so feasible for other countries. Of course, proper
tagging should be applied to these relations in order to don't create
ambiguities.

César


2013/10/3 Martin Koppenhoefer dieterdre...@gmail.com


 2013/10/3 Pieren pier...@gmail.com

 All the states or countries maps I've seen in my life used the
 coastline. It does not mean that the sovereignty stops at the water
 line. It's just a convention.



 how many states or country maps have you seen in scale 1:1000?

 cheers,
 Martin

 ___
 talk mailing list
 talk@openstreetmap.org
 https://lists.openstreetmap.org/listinfo/talk




-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-SIA: http://sia.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread César Martínez Izquierdo
That sounds interesting. I am not familiar with Nominatim, but I have
correctly understood, the result is a Postgres/postgis database with all
those polygons and hierarchies. This could be an interesting approach as
the post-processing could be directly done there using PostGIS predicates.

However, I am not sure about the generated hierarchies, as they don't keep
all OSM admin_levels into account (at least in the nomenclature: country,
state, county) and I see clear errors for Spain using the link [2] that you
provided. It would be interesting to know how these hierarchies are
generated (just OSM tags and geometric relations, external database, etc).

In any case, it seems a good starting point for my project.

Cheers,

Cesar


2013/10/3 Sarah Hoffmann lon...@denofr.de

 On Thu, Oct 03, 2013 at 12:32:36PM +0200, César Martínez Izquierdo wrote:
  Hi Frederik, regarding software, I am already familiar with Mapit scripts
  code, which are able to extract admin boundary polygons for each level
 (it
  is not creating relationships though). How do you see Nominatim or
  Osmium/osmjs better for the purpose?

 Nominatim already does a lot of the stuff you are planning to do. It
 creates geometries for admin boundaries from OSM data and puts them
 in a proper hierarchy. It is able to process updates and in the
 process makes sure that boundaries do not just disappear if somebody
 breaks the relation. If you only process the data that interests you
 (boundary=administrative and place nodes) it is not even that
 resource-hungry.

 It does have support for listing broken boundaries [1] and during the
 last hack day Brian has written a proof-of-concept for browsing admin
 hierarchies[2]. There is even a script to dump objects within a certain
 boundary[3] which could be easily extended to dump entire hierarchies.
 All these functions should currently be used with care though. There are
 known bugs and the output needs to be improved to make them really
 usable.

 Basically, most of the work to do would be on the output side, the
 heavy lifting of processing the data is already done. Well, apart
 from checking the OSM boundaries against reality. But I think wiki data
 is a good starting point for that.

  I will probably try the first option, but I expect that any of them would
  be costly to maintain if there are frequent changes of the tagging scheme
  for each country. (But nobody said it would be an easy task :-)

 Making boundary tagging more visible will hopefully help to stablize
 and unify the tagging schemas. The less country-specific exceptions
 the better.

 Sarah

 [1] http://nominatim.osm.org/polygons
 [2] http://nominatim.openstreetmap.org/hierarchy.php?place_id=97944985
 (This is for demonstration only, please do not scrape. It will not
 always give you the expected results anyways because there
 is a
 known bug with parenting which still lingers in the DB.)
 [3]
 https://github.com/lonvia/Nominatim/blob/export-script/utils/export.php

  2013/10/2 Frederik Ramm frede...@remote.org
 
   Hi,
  
   On 02.10.2013 18:23, César Martínez Izquierdo wrote:
I plan to create and make easily available a world-wide
 administrative
layer based on OSM data, ideally including existing administrative
 codes
(ISO, NUTS in Europe, etc) for each level and producing regular
 updates
(for instance once a year).
  
   This is something I have been thinking about for a long time (but never
   written any usable code).
  
   Nominatim is probably a good starting point - a better one than MapIt I
   should think.
  
   If you're only after extracting certain relation polygons then you
 could
   as well use osmjs (part of Osmium) and have it generate shape files for
   you, or adapt the shapefile/ogr export samples in Osmium; this will not
   yet give you a hierarchy, only individual boundaries, and you have to
   find out the hierarchy yourself.
  
   Finding out the hierarchy is going to be tricky. Nominatim does go to
   some lengths to do that already. It sounds easy (find all polygons
 with
   an admin level smaller than X where this polygon I'm looking at lies
   in). But in reality you will encounter at least:
  
   * missing polygons on all levels - sometimes simply not mapped,
   sometimes missing by design, e.g. Germany has some areas where admin
   levels 8, 6, and 4 coincide, these are mapped as admin_level 4, so draw
   a map of all admin_level 8 areas in Germany and you have lots of holes
   in them
  
   * broken polygons on all levels; brokenness changes by the day, i.e.
   what is working today may be broken tomorrow and vice versa
  
   * occasionally (e.g. Japan) linear regional boundaries that simply go
   from coast to coast without including the coastline
  
   * occasionally (e.g. Chile) a regional boundary that is not a
   multipolygon relation but instead a grouping of smaller regional
 entities
  
   * sometimes small geometric inaccuracies mean that 

Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread César Martínez Izquierdo
Thanks Eugene, that looks really promising.
I've seen there is an API to query Wikidata (results can be list of
Wikidata item IDs encoded as JSON), but I don't see the way to get the item
itself as JSON (or any other parseable format). Is it on the way?

César


2013/10/3 Eugene Alvin Villar sea...@gmail.com

 On Thu, Oct 3, 2013 at 6:35 PM, César Martínez Izquierdo 
 cesar@gmail.com wrote:

 Eugene, I am also interested on your proposal to store on Wikidata a
 table/database similar to the one described on 1, so any further details on
 available infrastructure, technologies in use, work already done, etc are
 welcome.


 Hi César, you can look at this Wikidata page for the German state of
 Baden-Württemberg as an example: https://www.wikidata.org/wiki/Q985

 It currently contains interesting properties and relationships that may be
 what we need. Some interesting properties/relations are (especially the OSM
 one):

 country=Germany
 capital=Stuttgart
 type of administrative division=state of Germany
 ISO 3166-2=DE-BW
 GND identifier=4004176-1
 contains administrative division=Regierungsbezirk Freiburg,
 Regierungsbezirk Karlsruhe, Regierungsbezirk Stuttgart, Regierungsbezirk
 Tübingen
 is in the administrative unit=Germany
 OpenStreetMap Relation ID=62611




-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-SIA: http://sia.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread Sarah Hoffmann
On Thu, Oct 03, 2013 at 03:31:20PM +0200, César Martínez Izquierdo wrote:
 That sounds interesting. I am not familiar with Nominatim, but I have
 correctly understood, the result is a Postgres/postgis database with all
 those polygons and hierarchies. This could be an interesting approach as
 the post-processing could be directly done there using PostGIS predicates.

Yes, exactly.

 However, I am not sure about the generated hierarchies, as they don't keep
 all OSM admin_levels into account (at least in the nomenclature: country,
 state, county) 

It keeps the levels internally and also uses the full levels for the
hierarchies. The levels are only grouped for output.

 and I see clear errors for Spain using the link [2] that you
 provided. It would be interesting to know how these hierarchies are
 generated (just OSM tags and geometric relations, external database, etc).

The hierarchies are built with OSM data only using the tagged admin_levels and
relating the polygon geometries. Basically, the parent is defined as the
area that covers the object that has the next smaller admin_level. It
gets a bit more complicated when place nodes are involved.

Sarah


 2013/10/3 Sarah Hoffmann lon...@denofr.de
 
  On Thu, Oct 03, 2013 at 12:32:36PM +0200, César Martínez Izquierdo wrote:
   Hi Frederik, regarding software, I am already familiar with Mapit scripts
   code, which are able to extract admin boundary polygons for each level
  (it
   is not creating relationships though). How do you see Nominatim or
   Osmium/osmjs better for the purpose?
 
  Nominatim already does a lot of the stuff you are planning to do. It
  creates geometries for admin boundaries from OSM data and puts them
  in a proper hierarchy. It is able to process updates and in the
  process makes sure that boundaries do not just disappear if somebody
  breaks the relation. If you only process the data that interests you
  (boundary=administrative and place nodes) it is not even that
  resource-hungry.
 
  It does have support for listing broken boundaries [1] and during the
  last hack day Brian has written a proof-of-concept for browsing admin
  hierarchies[2]. There is even a script to dump objects within a certain
  boundary[3] which could be easily extended to dump entire hierarchies.
  All these functions should currently be used with care though. There are
  known bugs and the output needs to be improved to make them really
  usable.
 
  Basically, most of the work to do would be on the output side, the
  heavy lifting of processing the data is already done. Well, apart
  from checking the OSM boundaries against reality. But I think wiki data
  is a good starting point for that.
 
   I will probably try the first option, but I expect that any of them would
   be costly to maintain if there are frequent changes of the tagging scheme
   for each country. (But nobody said it would be an easy task :-)
 
  Making boundary tagging more visible will hopefully help to stablize
  and unify the tagging schemas. The less country-specific exceptions
  the better.
 
  Sarah
 
  [1] http://nominatim.osm.org/polygons
  [2] http://nominatim.openstreetmap.org/hierarchy.php?place_id=97944985
  (This is for demonstration only, please do not scrape. It will not
  always give you the expected results anyways because there
  is a
  known bug with parenting which still lingers in the DB.)
  [3]
  https://github.com/lonvia/Nominatim/blob/export-script/utils/export.php
 
   2013/10/2 Frederik Ramm frede...@remote.org
  
Hi,
   
On 02.10.2013 18:23, César Martínez Izquierdo wrote:
 I plan to create and make easily available a world-wide
  administrative
 layer based on OSM data, ideally including existing administrative
  codes
 (ISO, NUTS in Europe, etc) for each level and producing regular
  updates
 (for instance once a year).
   
This is something I have been thinking about for a long time (but never
written any usable code).
   
Nominatim is probably a good starting point - a better one than MapIt I
should think.
   
If you're only after extracting certain relation polygons then you
  could
as well use osmjs (part of Osmium) and have it generate shape files for
you, or adapt the shapefile/ogr export samples in Osmium; this will not
yet give you a hierarchy, only individual boundaries, and you have to
find out the hierarchy yourself.
   
Finding out the hierarchy is going to be tricky. Nominatim does go to
some lengths to do that already. It sounds easy (find all polygons
  with
an admin level smaller than X where this polygon I'm looking at lies
in). But in reality you will encounter at least:
   
* missing polygons on all levels - sometimes simply not mapped,
sometimes missing by design, e.g. Germany has some areas where admin
levels 8, 6, and 4 coincide, these are mapped as admin_level 4, so draw
a map of all 

Re: [OSM-talk] Administrative boundaries export

2013-10-03 Thread Eugene Alvin Villar
On Fri, Oct 4, 2013 at 1:35 AM, César Martínez Izquierdo 
cesar@gmail.com wrote:

 Thanks Eugene, that looks really promising.
 I've seen there is an API to query Wikidata (results can be list of
 Wikidata item IDs encoded as JSON), but I don't see the way to get the item
 itself as JSON (or any other parseable format). Is it on the way?


Unfortunately, I am not up-to-par with the API side of Wikidata. I assume
that every bit of data on Wikidata can or will be accessible through APIs.
Otherwise, it would limit the usefulness of Wikidata if we resort to
scraping the HTML page.



 2013/10/3 Eugene Alvin Villar sea...@gmail.com

 On Thu, Oct 3, 2013 at 6:35 PM, César Martínez Izquierdo 
 cesar@gmail.com wrote:

 Eugene, I am also interested on your proposal to store on Wikidata a
 table/database similar to the one described on 1, so any further details on
 available infrastructure, technologies in use, work already done, etc are
 welcome.


 Hi César, you can look at this Wikidata page for the German state of
 Baden-Württemberg as an example: https://www.wikidata.org/wiki/Q985

 It currently contains interesting properties and relationships that may
 be what we need. Some interesting properties/relations are (especially the
 OSM one):

 country=Germany
 capital=Stuttgart
 type of administrative division=state of Germany
 ISO 3166-2=DE-BW
 GND identifier=4004176-1
 contains administrative division=Regierungsbezirk Freiburg,
 Regierungsbezirk Karlsruhe, Regierungsbezirk Stuttgart, Regierungsbezirk
 Tübingen
 is in the administrative unit=Germany
 OpenStreetMap Relation ID=62611


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


[OSM-talk] Administrative boundaries export

2013-10-02 Thread César Martínez Izquierdo
Hi,

I plan to create and make easily available a world-wide administrative
layer based on OSM data, ideally including existing administrative codes
(ISO, NUTS in Europe, etc) for each level and producing regular updates
(for instance once a year).

I think such a layer would be very useful for a number of scenarios, for
instance for scientific analysis involving administrative boundaries,
specially in areas where such data is not publicly available. Moreover, it
would probably attract collaborators to complete or correct the OSM
boundaries where needed.

I'd like to learn about similar initiatives (I don't want to replicate
efforts) or interested people, and also about existing tools for
simplifying the work. My first idea is to use Map-it scripts (which create
KML Admin Boundaries files from OSM data) as a basis for this work, but
other ideas are welcome.

Thanks in advance,

César Martínez Izquierdo

-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-SIA: http://sia.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-02 Thread Simone Cortesi
On Wed, Oct 2, 2013 at 6:23 PM, César Martínez Izquierdo
cesar@gmail.com wrote:
 I plan to create and make easily available a world-wide administrative layer
 based on OSM data, ideally including existing administrative codes (ISO,
 NUTS in Europe, etc) for each level and producing regular updates (for
 instance once a year).

thanks for jumping on this. I think this is a much needed export.

some thoughts:
1. admin data is stored with relations, which is not that easy to
maintain. you WILL always find broken borders.
2. having a web app to visualize that, and allow people to fix them is
something needed. similar to what we already have for the coastline.
3. once a year might be too much time between releases, think about a
once a year stable release, where all borders compile correctly.
4. show stats on broken, fixed borders.
5. provide them as a download in shapefile, poly format, geojson, and kml.

Thanks,
Simone.

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-02 Thread Frederik Ramm
Hi,

On 02.10.2013 18:23, César Martínez Izquierdo wrote:
 I plan to create and make easily available a world-wide administrative
 layer based on OSM data, ideally including existing administrative codes
 (ISO, NUTS in Europe, etc) for each level and producing regular updates
 (for instance once a year).

This is something I have been thinking about for a long time (but never
written any usable code).

Nominatim is probably a good starting point - a better one than MapIt I
should think.

If you're only after extracting certain relation polygons then you could
as well use osmjs (part of Osmium) and have it generate shape files for
you, or adapt the shapefile/ogr export samples in Osmium; this will not
yet give you a hierarchy, only individual boundaries, and you have to
find out the hierarchy yourself.

Finding out the hierarchy is going to be tricky. Nominatim does go to
some lengths to do that already. It sounds easy (find all polygons with
an admin level smaller than X where this polygon I'm looking at lies
in). But in reality you will encounter at least:

* missing polygons on all levels - sometimes simply not mapped,
sometimes missing by design, e.g. Germany has some areas where admin
levels 8, 6, and 4 coincide, these are mapped as admin_level 4, so draw
a map of all admin_level 8 areas in Germany and you have lots of holes
in them

* broken polygons on all levels; brokenness changes by the day, i.e.
what is working today may be broken tomorrow and vice versa

* occasionally (e.g. Japan) linear regional boundaries that simply go
from coast to coast without including the coastline

* occasionally (e.g. Chile) a regional boundary that is not a
multipolygon relation but instead a grouping of smaller regional entities

* sometimes small geometric inaccuracies mean that e.g. a state boundary
fails the is-in test for the country boundary because there's just a
little square metre somewhere that is mapped as belonging to the state
but not the country

* overlapping admin polygons of the same admin level

I think that ate the very least you need to run the evaluation regularly
and compare: Do I have new polygons this week - have others vanished,
and if so, is that because they were explicitly deleted/replaced, or
were they just accidentally broken and I should continue to use last week's?

What we would really need though, is something much bigger: A separate
database of admin hierarchies, where people could - in a crowdsourced
manner - record things like:

There is an adminlevel 2 entities called Germany
It is divided into 16 exclusive adminlevel 4 entities with the
following names: ...
These 16 entities cover the area of Germany completely (no holes or sea
areas that would be outside of one of the entities)
The adminlevel 4 entity named 'Brandenburg' is divided in X adminlevel
6 entities...

and so on. A tree of arbitrary size where people can add and edit at will.

Now you will say but this tree could be generated from OpenStreetMap,
and I grant that one could attempt to build such a tree but it will
always be faulty and reflect the current brokenness of geometries in
OSM. One could *start* with an OSM-generated tree, but after that, the
tree must be kept separate. People should be able to add stuff to the
tree even when it is not in OpenStreetMap - there should be an
adminlevel 8 boundary called so-and-so. A regularly-running process
would then compare the tree to OpenStreetMap, and generate error reports
that can be presented visually:

The tree says that there should be a region called X in Germany but OSM
doesn't have one.

There is an area here that is not covered by any adminlevel 4 area but
the tree says that taken together the adminlevel 4 areas must cover all
of the country.

The tree claims there should be a region called X but in OSM there's
only a region with the similar name Y, which one is correct?

and so on. - I would expect the tree to be much more stable than the
data in OSM. Most of all, the tree could be worked on independently,
even by people unfamiliar with OSM. Of course the tree could link to OSM
objects but these links would regularly be checked and perhaps even
changed by the automated comparison system.

As I said, a simple export is easy but it will have too many weaknesses
- you couldn't even say to what level a country is complete. Some
people have started to link regional boundaries in OSM together with
concepts like subarea but I don't think this can replace an external
country structure tree because the tree could describe what is expected
to be there whereas in OSM you never know if some thing is missing
because it doesn't exist (cf. my example about the missing admin6/8
boundaries in Germany) or if it just hasn't been mapped yet, or is broken.

It would be great to see someone drive this important topic but it
certainly isn't something you can set up in a week or two ;)

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09 E008°23'33


Re: [OSM-talk] Administrative boundaries export

2013-10-02 Thread Eugene Alvin Villar
On Thu, Oct 3, 2013 at 3:58 AM, Frederik Ramm frede...@remote.org wrote:

 What we would really need though, is something much bigger: A separate
 database of admin hierarchies, where people could - in a crowdsourced
 manner - record things like:

 There is an adminlevel 2 entities called Germany
 It is divided into 16 exclusive adminlevel 4 entities with the
 following names: ...
 These 16 entities cover the area of Germany completely (no holes or sea
 areas that would be outside of one of the entities)
 The adminlevel 4 entity named 'Brandenburg' is divided in X adminlevel
 6 entities...

 and so on. A tree of arbitrary size where people can add and edit at will.


This seems exactly like the kind of data I expect Wikidata (a Wikimedia
Foundation project, spearheaded by Wikimedia Deutschland) to encode.


 Now you will say but this tree could be generated from OpenStreetMap,
 and I grant that one could attempt to build such a tree but it will
 always be faulty and reflect the current brokenness of geometries in
 OSM. One could *start* with an OSM-generated tree, but after that, the
 tree must be kept separate. People should be able to add stuff to the
 tree even when it is not in OpenStreetMap - there should be an
 adminlevel 8 boundary called so-and-so. A regularly-running process

would then compare the tree to OpenStreetMap, and generate error reports
 that can be presented visually: [...]

 I would expect the tree to be much more stable than the
 data in OSM. Most of all, the tree could be worked on independently,
 even by people unfamiliar with OSM. Of course the tree could link to OSM
 objects but these links would regularly be checked and perhaps even
 changed by the automated comparison system.


If this tree database will only be compared to OSM then it should be OK to
start it as a derivative of the OSM database (inheriting the ODbL license).
However, we lose the ability to compare this with other similar databases
like the Global Administrative Areas database (http://www.gadm.org/) as
another form of QA due to the license incompatibility.

I think it would be better if the tree were started in Wikidata (which is
CC0-licensed) or as a separate project released to the public domain or
CC0- or PDDL-licensed.

Eugene
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-02 Thread Dale Kunce
The American Red Cross is dealing with this issue right now.

We've built a database using GADM, GAUL, and Natural Earth. We initially
wanted to rely heavily on OSM but because the relation breaks said it
proved problematic. We've parked the use of OSM in our boundary dataset so
that we can move forward with our implementation. We would love to help
build out a plausible solution if one can be developed.

The basic features of our current in production solution:

   - Name based lookup for anything in GADM, GAUL, Natural Earth, and
   GeoNames.
   - Ability to get the admin stack, all admin boundaries above the
   currently selected feature, geoname, or WKT.
   - Ability to look at all of the datasets through time to use GADM.
   - Ability to get any feature from any of the datasets in geoJSON format.


Our roadmap includes an API link to Nominatim and using the OSM polygon
boundaries to then parse against our internal system to get the admin
stack names and then going back to Nominatim to get the OSM geometry for
the given features. Not the most elegant solution but I think it should
work for most of our work where we want to use OSM admin boundaries.

The code for our project is available on our github page.
https://github.com/AmericanRedCross/GeoWebServices and
https://github.com/AmericanRedCross/GeoDB.




On Wed, Oct 2, 2013 at 4:01 PM, Eugene Alvin Villar sea...@gmail.comwrote:

 On Thu, Oct 3, 2013 at 3:58 AM, Frederik Ramm frede...@remote.org wrote:

 What we would really need though, is something much bigger: A separate
 database of admin hierarchies, where people could - in a crowdsourced
 manner - record things like:

 There is an adminlevel 2 entities called Germany
 It is divided into 16 exclusive adminlevel 4 entities with the
 following names: ...
 These 16 entities cover the area of Germany completely (no holes or sea
 areas that would be outside of one of the entities)
 The adminlevel 4 entity named 'Brandenburg' is divided in X adminlevel
 6 entities...

 and so on. A tree of arbitrary size where people can add and edit at will.


 This seems exactly like the kind of data I expect Wikidata (a Wikimedia
 Foundation project, spearheaded by Wikimedia Deutschland) to encode.


 Now you will say but this tree could be generated from OpenStreetMap,
 and I grant that one could attempt to build such a tree but it will
 always be faulty and reflect the current brokenness of geometries in
 OSM. One could *start* with an OSM-generated tree, but after that, the
 tree must be kept separate. People should be able to add stuff to the
 tree even when it is not in OpenStreetMap - there should be an
 adminlevel 8 boundary called so-and-so. A regularly-running process

 would then compare the tree to OpenStreetMap, and generate error reports
 that can be presented visually: [...]


 I would expect the tree to be much more stable than the
 data in OSM. Most of all, the tree could be worked on independently,
 even by people unfamiliar with OSM. Of course the tree could link to OSM
 objects but these links would regularly be checked and perhaps even
 changed by the automated comparison system.


 If this tree database will only be compared to OSM then it should be OK to
 start it as a derivative of the OSM database (inheriting the ODbL license).
 However, we lose the ability to compare this with other similar databases
 like the Global Administrative Areas database (http://www.gadm.org/) as
 another form of QA due to the license incompatibility.

 I think it would be better if the tree were started in Wikidata (which is
 CC0-licensed) or as a separate project released to the public domain or
 CC0- or PDDL-licensed.

 Eugene

 ___
 talk mailing list
 talk@openstreetmap.org
 https://lists.openstreetmap.org/listinfo/talk




-- 
sent from my mobile device

Dale Kunce
http://normalhabit.com
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-02 Thread Christian Quest
Frederik explained many of the usual troubles you may encounter. Sorry to
had many a few more :(

I recently had to deal with admin boundaries and the lack of homogeneous
tagging of worldwide reference numbers (like iso3166 or FIPS) does not help.
For example US states had only the usual 2 letters in the ref=* tag, no
fips code (which is a US federal classification) nor iso3166... so I added
fips codes to US states.

We could already start by making sure these worldwide references are
present in OSM data, and do some QA on them (make sure none is missing,
valid polygons, etc).

To help you, have a look at http://layers.openstreetmap.fr/ which renders
admin_level from 2 to 10...

See for example level 4 in Europe:
http://layers.openstreetmap.fr/?zoom=6lat=48.21736lon=2.70236layers=0B000T

UK level 4 is on the maritime borders (island culture ?) where most other
European countries stop on the coastline... tagging bio-diversity is not
helpful !



2013/10/2 César Martínez Izquierdo cesar@gmail.com

 Hi,

 I plan to create and make easily available a world-wide administrative
 layer based on OSM data, ideally including existing administrative codes
 (ISO, NUTS in Europe, etc) for each level and producing regular updates
 (for instance once a year).

 I think such a layer would be very useful for a number of scenarios, for
 instance for scientific analysis involving administrative boundaries,
 specially in areas where such data is not publicly available. Moreover, it
 would probably attract collaborators to complete or correct the OSM
 boundaries where needed.

 I'd like to learn about similar initiatives (I don't want to replicate
 efforts) or interested people, and also about existing tools for
 simplifying the work. My first idea is to use Map-it scripts (which create
 KML Admin Boundaries files from OSM data) as a basis for this work, but
 other ideas are welcome.

 Thanks in advance,

 César Martínez Izquierdo

 --
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
César Martínez Izquierdo
GIS developer
-  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
ETC-SIA: http://sia.eionet.europa.eu/
Universitat Autònoma de Barcelona (SPAIN)
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

 ___
 talk mailing list
 talk@openstreetmap.org
 https://lists.openstreetmap.org/listinfo/talk




-- 
Christian Quest - OpenStreetMap France
Un nouveau serveur pour OSM... http://donate.osm.org/server2013/
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Administrative boundaries export

2013-10-02 Thread Eugene Alvin Villar
On Thu, Oct 3, 2013 at 6:52 AM, Christian Quest cqu...@openstreetmap.frwrote:

 UK level 4 is on the maritime borders (island culture ?) where most other
 European countries stop on the coastline... tagging bio-diversity is not
 helpful !


This is actually another point to consider when extracting admin boundaries
from OSM data.

My personal view is that the admin boundary marks the boundary where the
administrative entity exercises jurisdiction. Under UNCLOS, nations are
allowed to exercise full sovereignty over internal waters (which includes
water seaward of the coastline but within the baselines) and almost full
sovereignty (foreign ships are allowed innocent passage) for territorial
waters (up to 12 nautical miles from the baselines). So I think that using
the maritime boundaries as the admin_level=2 boundary is not incorrect and
this is reflected in OSM.

Use of maritime boundaries for admin_level=3 and higher (such as with UK)
depends on how the particular nation interprets its internal
maritime/fisheries laws. In my country, we have municipal waters which
can be up to 15 kilometers from the shoreline and that's where
municipalities can exercise jurisdiction.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk