Re: [OSM-talk] osm2pgsql hstore (was: Wind turbines no longer rendered on mapnik layer)
Frederik Ramm remote.org> writes: > > Hi, > > On 02/16/2012 07:25 PM, Graham Jones wrote: > > This reminded me of a question I have been meaning to ask for quite a > > while - is a database generated by osm2pgsql with an hstore expected to > > perform similarly to one without? > > I never ran one with hstore when I think of what this must mean for the > database engine, and storage space, then I shudder and would not be > surprised by the factor 5 you mentioned. How about doing it with relations? Let the importer program create four tables osm_point osm_line osm_polygon osm_relation These tables would each have two attributes: Geometry and osm_is. Geometries in the osm_relation would be of type "geometry collection", that is, a collection of whatever, and it could hold for example the tranport route relation with the route and bus stops and everything in one PostGIS geometry object. Then there should be one or four tables for tags (everything in one table or split to suit the geometry tables). The three attributes would be osm_id, key, and value. Osm_id would be used as a foreing key for joining geometries and attributes. If osm_id can not be guaranteed to be unique then there should be point_id, line_id, polygon_id and relation_id added to suitable places. Every attribute would be indexed. For Mapnik use where all that is needed is to do simple SQL selects. I guess that this database would be faster to query than hstore even it also contains all the tags. It might be faster for Mapnik than the current tables with extra wide attribute schema because now attributes are not indexed at all (or are they?). SQL queries according to tags and values or even with a part of the tag and value strings would be easier to do than from hstore. People could enhance attributes for some special uses by converting max_speeds and other values which are actually measures from strings to integers or doubles etc. This option is so obvious that I believe that someone must have tried it already. It would be nice to hear about experiences. -Jukka Rahkonen- ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] osm2pgsql hstore (was: Wind turbines no longer rendered on mapnik layer)
2012/2/17 Jochen Topf : > On Thu, Feb 16, 2012 at 06:25:30PM +, Graham Jones wrote: > Generic key names can be confusing, especially when one OSM object is used for > multiple things. +1. E.g. an object tagged barrier=fence, height=2, landuse=forest. In this example you could also use 2 objects: a closed way for the fence and a multipolygon for the area. Linear objects we can create with route relations (instead of overlapping ways). We don't have currently a relation for nodes and maybe we don't need this utterly (you could place another node "nearby"), but it would solve some cases where you want to state the topology precisely (e.g. 2 objects at the same pole). This way we can have "things" (expressed by a relation) which have their geometry only as an attribute, instead of a geometry that _is_ something. > Non-generic tags also make tools such as Taginfo more useful, because each tag > stands on its own. They make it immediately clear if the tagging is > incomplete. > If the "generator" tag got lost, how do we know what the "power_source" tag > is supposed to mean? +1 for taginfo, but I don't think we need the namespaces for the case something gets "lost" (there is the history for this). Cheers, Martin ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
[OSM-talk] Typo in Trac "browse by components" fixed
Hi, Just noticed that on the browse by components page (http://trac.openstreetmap.org/wiki/OsmComponentReports), every link had a typo. eg http://trac.openstreetmap.org/query?status=new&status=assigend&status=reopened&component=potlatch2&order=priority should be http://trac.openstreetmap.org/query?status=new&status=assigned&status=reopened&component=potlatch2&order=priority (assigend -> assigned) Fixed now. (In case anyone is wondering why they're suddenly seeing a few more tickets than previously) Steve ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] osm2pgsql hstore (was: Wind turbines no longer rendered on mapnik layer)
On Thu, Feb 16, 2012 at 06:25:30PM +, Graham Jones wrote: > Why create a key generator:power_source rather than just use > power_source. power_source is much more generic so you could re-cycle it > for things like district heating, but generator:power_source is only ever > going to be used for generating stations, and needs a new column in the > database. . I think I just prefer more generic, re-usable keys > rather than trying to invent a new one for each situation Generic key names can be confusing, especially when one OSM object is used for multiple things. Say there is a way tagged as railway and at the same time this way is part of an area tagged as a generating station. Does "power_source" mean the type of generating station or the type of power used by the railway (overhead vs. third rail vs. unelectrified)? I hope this example is hypothetical, but people do strange things in OSM... Non-generic tags also make tools such as Taginfo more useful, because each tag stands on its own. They make it immediately clear if the tagging is incomplete. If the "generator" tag got lost, how do we know what the "power_source" tag is supposed to mean? So on the one side you have generic tags like "type" and "class" and "id" which are very confusing. You never know what they belong to. On the other side you have rather complex tagging schemas like in your generator:... example. Thats sometimes hard to use, too. We still have to figure out where the sweet spot is, but I tend to prefer the more explicit keys. Jochen -- Jochen Topf joc...@remote.org http://www.remote.org/jochen/ +49-721-388298 ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-legal-talk] Some numbers on split ways
Hi, I thought some extra numbers could be interesting: # of ways that turned from tainted to clean: 11 # of ways that turned from clean to tainted: 216 total # of detected split ways: 1856 cheers ant On 16.02.2012 12:40, ant wrote: Hi, although I'm late - I want to present some "hard numbers" on the effect of split ways. I've written a script that is based on the (imprecise) assumption that a way has resulted from a split if its version 1 has 2 or more nodes that another way from the same changeset has had before. Also such ways are discovered only if a way (pre-split) has lost at least 3 nodes from one version to another. So naturally the actual number of splits might differ from the number of splits the script detects. Then if a pre-split way is found to be the ancestor of another way, all versions that were created before the split are appended before the new way's version 1, with version numbers (-n ... 0), carrying original nodes and tags. On these extended history objects the licence status was evaluated with conventional WTFE logic. Input data was a small (city-sized) history extract, dated 2011-11-19.[1] Agreed users data and WTFE code were from 2012-01-16.[2] Total number of ways: 17273 Number of ways created by denier - without split detection: 1129 (incl. 227 deleted) - with split detection: 1274 (incl. 234 deleted) Number of ways modified by denier[4] - without split detection: 381 (incl. 8 deleted) - with split detection: 441 (incl. 10 deleted) There's still lots of work to be done to make it read larger data chunks (currently the script reads and writes XML and is really, really slow)... I'll make the scripts available if anyone is interested in taking up the work. I'm not going to have much time for this during the next weeks. cheers ant [1] http://ftp5.gwdg.de/pub/misc/openstreetmap/osm-full-history-extracts/110919/xml/europe/germany/rheinland-pfalz/mainz.osh.bz2 [2] http://planet.openstreetmap.org/users_agreed/users_agreed.txt http://planet.openstreetmap.org/users_agreed/anon_changesets_agreed.txt http://wtfe.gryph.de/report.tgz (no user or changeset overrides) [4] Ways affected only by harmless edits are not counted. ___ legal-talk mailing list legal-t...@openstreetmap.org http://lists.openstreetmap.org/listinfo/legal-talk
Re: [OSM-talk] Wind turbines no longer rendered on mapnik layer
On 15-2-2012 17:06, Morten Kjeldgaard wrote: I received the following reply at 10.14.44 GMT+01:00: Hi mok0, Thanks for the notice, and I'm currently submitting a changeset where everything with generator:source=wind also has power_source=wind specified as well. Also, I'll take a look into the mapnik style sheet to see how this can be updated to remove the depreciated tags. Aside from the existence of generator:source=wind this demonstrates he hasn't quite understood it yet. The new scheme should be added[1] to the stylesheet, but the old scheme should not be removed. At least not before the overwhelming majority of existing objects are compatible with the new tagging, or after some extensive time has passed to allow this tag change to propagate. [1] Has now been done, but not deployed yet. -- Lennard ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] osm2pgsql hstore (was: Wind turbines no longer rendered on mapnik layer)
On 16-2-2012 19:25, Graham Jones wrote: Why create a key generator:power_source rather than just use power_source. power_source is much more generic so you could re-cycle it for things like district heating, but generator:power_source is only ever going to be used for generating stations, and needs a new column in the database. . I think I just prefer more generic, re-usable keys rather than trying to invent a new one for each situation This is a core problem with the public_transport=* scheme as well. This tag in and of itself is fine, but the whole additional shebang that is train=yes/no, bus=yes/no, subway=yes/no, etc etc is so dreadful (in the context of the rendering chain) that every time I read #2798 I feel the urge to run away screaming. http://trac.openstreetmap.org/ticket/2798 -- Lennard ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] osm2pgsql hstore (was: Wind turbines no longer rendered on mapnik layer)
On 16 February 2012 18:51, Frederik Ramm wrote: > On 02/16/2012 07:25 PM, Graham Jones wrote: > >> This reminded me of a question I have been meaning to ask for quite a >> while - is a database generated by osm2pgsql with an hstore expected to >> perform similarly to one without? >> > > I never ran one with hstore when I think of what this must mean for the > database engine, and storage space, then I shudder and would not be > surprised by the factor 5 you mentioned. Thanks - I don't feel so bad about giving up so easily now! If you want to add a new key but avoid re-importing the full database, you > can add the key to osm2pgsql's style file, somehow generate an .osm file > that contains only the objects that have this tag, wrap this .osm file into > a "..." instead of > "..." and throw it at osm2pgsql in append mode. That's a neat idea - will try that next time, thanks. > People like keys that are understandable without context. Contrast this > with a tag "width=2" - where you have no chance in knowing what it means > without looking at the whole thing. Yes - I think it is just a style difference - I am quite happy to interpret it from context for the sake of having a smaller number of unique keys to think about. Cheers Graham -- Graham Jones Hartlepool, UK. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] osm2pgsql hstore (was: Wind turbines no longer rendered on mapnik layer)
Hi, On 02/16/2012 07:25 PM, Graham Jones wrote: This reminded me of a question I have been meaning to ask for quite a while - is a database generated by osm2pgsql with an hstore expected to perform similarly to one without? I never ran one with hstore when I think of what this must mean for the database engine, and storage space, then I shudder and would not be surprised by the factor 5 you mentioned. I keep having to re-import my database to add new keys. If you want to add a new key but avoid re-importing the full database, you can add the key to osm2pgsql's style file, somehow generate an .osm file that contains only the objects that have this tag, wrap this .osm file into a "..." instead of "..." and throw it at osm2pgsql in append mode. power_source is much more generic so you could re-cycle it for things like district heating, but generator:power_source is only ever going to be used for generating stations, and needs a new column in the database. Yes, that's the plan I believe. People like keys that are understandable without context. Contrast this with a tag "width=2" - where you have no chance in knowing what it means without looking at the whole thing. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Keeping tags generic (was: Wind turbines no longer rendered on mapnik layer)
On Thu, Feb 16, 2012 at 6:25 PM, Graham Jones wrote: > Why create a key generator:power_source rather than just use > power_source. power_source is much more generic so you could re-cycle it > for things like district heating, but generator:power_source is only ever > going to be used for generating stations, and needs a new column in the > database. . I think I just prefer more generic, re-usable keys > rather than trying to invent a new one for each situation Seconded heartily! ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
[OSM-talk] osm2pgsql hstore (was: Wind turbines no longer rendered on mapnik layer)
On 16 February 2012 16:25, Frederik Ramm wrote: > Hi, > > > On 02/16/2012 03:05 PM, Kay Drangmeister wrote: > >> Isn't there a hstore in the rendering-db? >> > > No. > > This reminded me of a question I have been meaning to ask for quite a while - is a database generated by osm2pgsql with an hstore expected to perform similarly to one without? The only time I tried to use hstore, the performance of the whole database was dreadful (at least a factor of 5 slower, but I didn't do any proper benchmarking - I just gave up and re-created it without hstore and the problem went away).It may have been that something went wrong with the import that I did not notice, and it was not an hstore issue. Using hstore would be really good given the fashion for generating new keys with colons in them, but not with that reduction in performance, so I keep having to re-import my database to add new keys. Why create a key generator:power_source rather than just use power_source. power_source is much more generic so you could re-cycle it for things like district heating, but generator:power_source is only ever going to be used for generating stations, and needs a new column in the database. . I think I just prefer more generic, re-usable keys rather than trying to invent a new one for each situation Regards Graham. -- Graham Jones Hartlepool, UK. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Finding untagged dead-ends
John Sturdy wrote > > Something I'd like to have a way of highlighting, as I've been doing > quite a bit of electricity distribution mapping, is power lines / > minor_lines that end in something other than a transformer or pole. > FYI http://wiki.openstreetmap.org/wiki/Osmose has a test for this. Unfortunetly, the tool only covers France but is free. -- View this message in context: http://gis.19327.n5.nabble.com/Finding-untagged-dead-ends-tp5485340p5490081.html Sent from the General Discussion mailing list archive at Nabble.com. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wind turbines no longer rendered on mapnik layer
Hi, On 02/16/2012 03:05 PM, Kay Drangmeister wrote: Isn't there a hstore in the rendering-db? No. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wind turbines no longer rendered on mapnik layer
Vincent Privat wrote: So maybe we'll finally have an answer to our questions in http://trac.openstreetmap.org/ticket/3266 ? Isn't there a hstore in the rendering-db? Kay ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wind turbines no longer rendered on mapnik layer
So maybe we'll finally have an answer to our questions in http://trac.openstreetmap.org/ticket/3266 ? ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
