Re: [OSM-dev] Turning boundaries to linestrings

[Paul Norman]

On 12/13/2016 9:04 AM, Martin Koppenhoefer wrote:
that's a nice hack which will generally work, but having overlapping 
boundaries (different ways on the same position) is not a clear 
"error" in OSM, so it is not 100% reliable.


No. I looked at the data visually with transparency to find overlaps, 
and they are exceedingly rare. As a data consumer, I have no hesitation 
in calling what you have described an error.


___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Turning boundaries to linestrings

[Paul Norman]

On 12/13/2016 4:33 AM, Christoph Hormann wrote:

On Tuesday 13 December 2016, Paul Norman wrote:

Feedback from anyone interested in using this output is welcome, as
well as any additional information that should be added to the
linestrings.

You have some conflicting goals here - you want to maintain the OSM
tagging information on the way level so you need to maintain the
individual ways - which however limits what you can do in terms of
rendering - dashing is not consistent for example where a boundary is
split and you have problems doing geometry processing (like the all
popular ST_Simplify()).


Maintaining the individual ways is not a goal. The primary goal is being 
able to render disputed boundaries, which this achieves, although not 
perfectly. The dashing problems are most visible comparing dashed 
borders on straight line borders in remote Canada vs several US states. 
In the former, the border tends to be one long linestring because it's 
unsplit and doesn't have many junction points, while in US states 
junction points are more common.



IMO a process that merges the ways where it is non-ambiguous would be
more useful.  And most cases where tagging varies in an admin boundary
without there being a junction that is a tagging error.


Most places where two admin boundary ways join are either part of a very 
long stretch (e.g. BC-AB border) where merging isn't a huge issue, or 
junction points of some kind where >2 admin boundary ways join. The 
second poses a problem for merging and simplification, which can be 
solved in a three ways


1. Merging on linestring admin_level and simplification by amounts that 
don't visibly break topology, even if examining the data on a sub-pixel 
accuracy shows breaks. More than this is a problem, not because of 
opening up small gaps, but having ways appear to extend to far. The 
former will end up interpreted as part of the dashes most of the time, 
but the latter can look odd.


2. Doing merging that varies based on rendered admin_levels. This would 
need to do a topology-aware simplification based on the admin_levels 
that are being rendered. If done in preprocessing by osmborder, this 
would probably need to result in 4-6 files being generated, each going 
from admin_level 2 to a different maximum.


3. Simplification without merging. This can cause topology problems, but 
not at junction points. Unfortunately, this won't help much with data 
volume or dashes


Given current priorities, I can't see putting topology awareness into 
osmborder or doing it at query time. If that happens, it will probably 
be when working on generated vector tile size for low-bandwidth devices. 
It's probably going to be simplification without merging at query time 
initially, and at some point I'll look at linestring merging for roads, 
and boundaries will be easy to do after that.



The maritime attribute based on tagging is currently of very limited use
since it is not consistently applied.  On admin_level 2 a better way
would be to classify this based on topology - i.e. all boundaries that
are only part of one admin_level 2 relation are maritime boundaries.
On the higher admin levels this is more complicated of course and not
easy to solve.


Yes, I've been considering if I should do something like that by 
counting number of parent relations, but not because of incomplete data. 
The data is pretty good for admin_level=2, and less consistent for 4.


The question I was considering is how I want to render boundaries in 
places like the Georgia Straight or Great Lakes, e.g. 
https://cloud.githubusercontent.com/assets/1190866/21168186/faeecaae-c166-11e6-8cd1-7488fffddca4.png. 
If I want those to be rendered, I have to count number of parent 
relations and style based on that.


___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Tilemaker - missing areas

[Michał Brzozowski]

On Tue, Dec 13, 2016 at 5:21 PM, Richard Fairhurst  wrote:
> Out of interest, what happens if you change the maxzoom and basezoom
> settings to 14 (from 16)?

Still a similar thing happens, there are maybe less dropped areas, but
they do happen often.
What's interesting is warning messages like "Exception when writing
output object 4294962007 of type 2". There's no such way or relation
and that number is suspiciously close to 2^32.

Michał

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Turning boundaries to linestrings

[Martin Koppenhoefer]

2016-12-13 13:33 GMT+01:00 Christoph Hormann :

> On admin_level 2 a better way
> would be to classify this based on topology - i.e. all boundaries that
> are only part of one admin_level 2 relation are maritime boundaries.
>


that's a nice hack which will generally work, but having overlapping
boundaries (different ways on the same position) is not a clear "error" in
OSM, so it is not 100% reliable.

Cheers,
Martin
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Tilemaker - missing areas

[Richard Fairhurst]

Michał Brzozowski wrote:
> Am I doing something silly or is it a bug in Tilemaker?

That looks like a bug in the Tilemaker code that calculates which areas fall
into which tiles. I'll have a look but if you could put this as a github
issue that'd be helpful.

Out of interest, what happens if you change the maxzoom and basezoom
settings to 14 (from 16)?

cheers
Richard



--
View this message in context: 
http://gis.19327.n8.nabble.com/Tilemaker-missing-areas-tp5887234p5887436.html
Sent from the Developer Discussion mailing list archive at Nabble.com.

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Turning boundaries to linestrings

[Christoph Hormann]

On Tuesday 13 December 2016, Paul Norman wrote:
>
> Feedback from anyone interested in using this output is welcome, as
> well as any additional information that should be added to the
> linestrings.

You have some conflicting goals here - you want to maintain the OSM 
tagging information on the way level so you need to maintain the 
individual ways - which however limits what you can do in terms of 
rendering - dashing is not consistent for example where a boundary is 
split and you have problems doing geometry processing (like the all 
popular ST_Simplify()).

IMO a process that merges the ways where it is non-ambiguous would be 
more useful.  And most cases where tagging varies in an admin boundary 
without there being a junction that is a tagging error.

The maritime attribute based on tagging is currently of very limited use 
since it is not consistently applied.  On admin_level 2 a better way 
would be to classify this based on topology - i.e. all boundaries that 
are only part of one admin_level 2 relation are maritime boundaries.  
On the higher admin levels this is more complicated of course and not 
easy to solve.

-- 
Christoph Hormann
http://www.imagico.de/

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] OpenStreetMap.org site statistics

[Tom Hughes]

On 13/12/16 11:50, Darafei "Komяpa" Praliaskouski wrote:


вт, 13 дек. 2016 г. в 13:55, Tom Hughes >:

On 13/12/16 10:33, Darafei "Komяpa" Praliaskouski wrote:

> It lists awstats for www, planet, and wiki. They all are down.

Interesting. I wasn't aware that had ever been public.

I think it does still try to run but frankly it's pretty hopeless with
the volume of data we have I think.


Looking
at https://github.com/openstreetmap/chef/tree/master/cookbooks/stats -
it should be public. No auth attempts that I see in configs.


I just meant I wasn't aware it was, not that it wasn't meant to be.

I think that was originally setup before I was involved, which shows how 
long ago it was ;-)



Â

> What is current way to look at statistics on osm.org
 
> visits, planet accesses and wiki popularity?

I don't think there are any public statistics.

Â
Are there non-public statistics?
Ghostery extension suggests there is Piwik installation on osm.org
.


Yes we are running piwik.


It's generally a tricky thing to do because of the privacy issues
involved and the need to make sure things are aggregated into big enough
buckets to avoid revealing anything we shouldn't.

What exactly and why shouldn't be revealed?


Well anything that can identify an individual, which might include 
aggregated values if buckets are too small.


Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] OpenStreetMap.org site statistics

[Komяpa]

вт, 13 дек. 2016 г. в 13:55, Tom Hughes :

> On 13/12/16 10:33, Darafei "Komяpa" Praliaskouski wrote:
>
> > It lists awstats for www, planet, and wiki. They all are down.
>
> Interesting. I wasn't aware that had ever been public.
>
> I think it does still try to run but frankly it's pretty hopeless with
> the volume of data we have I think.
>

Looking at https://github.com/openstreetmap/chef/tree/master/cookbooks/stats -
it should be public. No auth attempts that I see in configs.


> > What is current way to look at statistics on osm.org 
> > visits, planet accesses and wiki popularity?
>
> I don't think there are any public statistics.
>

Are there non-public statistics?
Ghostery extension suggests there is Piwik installation on osm.org.

It's generally a tricky thing to do because of the privacy issues
> involved and the need to make sure things are aggregated into big enough
> buckets to avoid revealing anything we shouldn't.
>

What exactly and why shouldn't be revealed?
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] OpenStreetMap.org site statistics

[Tom Hughes]

On 13/12/16 10:33, Darafei "Komяpa" Praliaskouski wrote:


It lists awstats for www, planet, and wiki. They all are down.


Interesting. I wasn't aware that had ever been public.

I think it does still try to run but frankly it's pretty hopeless with 
the volume of data we have I think.



What is current way to look at statistics on osm.org 
visits, planet accesses and wiki popularity?


I don't think there are any public statistics.

It's generally a tricky thing to do because of the privacy issues 
involved and the need to make sure things are aggregated into big enough 
buckets to avoid revealing anything we shouldn't.


Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

[OSM-dev] OpenStreetMap.org site statistics

[Komяpa]

Hello,

I'm looking at https://wiki.openstreetmap.org/wiki/Stats page.

It lists awstats for www, planet, and wiki. They all are down.

What is current way to look at statistics on osm.org visits, planet
accesses and wiki popularity?
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Main database wiki page

[Komяpa]

Thanks a lot.
I've updated the wiki page based on information from your e-mail.
https://wiki.openstreetmap.org/wiki/Database

пн, 12 дек. 2016 г. в 17:17, Andy Allan :

> On 12 December 2016 at 11:54, Darafei "Komяpa" Praliaskouski
>  wrote:
>
> > I've been googling for current human-readable document that describes
> what
> > OSM database currently is.
>
> As far as I'm aware, there's no up-to-date human-readable description
> of the primary OSM database. The database contains 40+ tables, and a
> bunch of foreign keys, sequences, and of course, many columns, so
> maintaining any documentation is not a trivial task. That's likely why
> any documentation you find is unlikely to have been kept up to date
> over the years.
>
> > Can someone with deeper knowledge update this page, and/or make it point
> to
> > a better place to read about current OSMF database cluster setup?
>
> There are two aspects to the database - A) the internal structure
> (tables, columns etc) and B) the production cluster setup
> (replication, backups etc)
>
> A) The internal structure is governed by the openstreetmap-website
> code base, specifically the db/structure.sql definition.
>
>
> https://github.com/openstreetmap/openstreetmap-website/blob/master/db/structure.sql
>
> This is not very human-friendly, but it is accurate. I never use this
> myself, since I find it easier to read the definitions in app/models/
> to find how the different tables relate to each other, and I use the
> postgresql command line to find out the details of the structure, if
> necessary.
>
> B) The cluster configuration is managed via the Chef configuration, found
> at:
>
> https://github.com/openstreetmap/chef
>
> What happens on each machine is defined in the roles, and the set-up
> of database-related machines is controlled by the "db" cookbook.
>
> https://github.com/openstreetmap/chef/tree/master/roles
> https://github.com/openstreetmap/chef/tree/master/cookbooks/db
>
> Again, these are not particularly human-readable, but since they are
> the real code that controls the configuration, they are always
> accurate. Again, I don't usually read the role files directly, but I
> would use https://hardware.openstreetmap.org/ , which is generated
> from the Chef configuration automatically, as a starting point and a
> more human-friendly approach.
>
> Please let me know if you have any questions and I'll do what I can to
> help.
>
> Thanks,
> Andy
>
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev