Re: [OSM-dev] Planet file Import OOM kills postgres

2011-05-24 Thread Frederik Ramm
Hi, Stefan Menzel wrote: i am trying to import the Planetfile into postgis. The machine has 8Gb RAM and i spent 4GB for osm2psql. I have observed, that osm2psql actually used 6 GB and postgres up to 1,8 Gb before it crashed. Which version of osm2pgsql did you use, and what was the exact comm

[OSM-dev] Planet file Import OOM kills postgres

2011-05-24 Thread Stefan Menzel
Hello, i am trying to import the Planetfile into postgis. The machine has 8Gb RAM and i spent 4GB for osm2psql. I have observed, that osm2psql actually used 6 GB and postgres up to 1,8 Gb before it crashed. Syslog tells Out of memory: kill process 1527 (postgres) score 104144 or a childe Ki

Re: [OSM-dev] osm2pgsql for 64-bit IDs

2011-05-24 Thread Igor Brejc
On Tue, May 24, 2011 at 5:25 PM, Frederik Ramm wrote: > Hi, > > > On 05/24/11 17:05, Igor Brejc wrote: > >> Not necessarily the same thing. Writing protobuf reader is still much >> easier than implementing something like >> http://www.sqlite.org/fileformat.html and >> http://www.sqlite.org/filefo

Re: [OSM-dev] osm2pgsql for 64-bit IDs

2011-05-24 Thread Igor Brejc
On Tue, May 24, 2011 at 5:09 PM, Peter Körner wrote: > Am 24.05.2011 17:05, schrieb Igor Brejc: > > Not necessarily the same thing. Writing protobuf reader is still much >> easier than implementing something like >> http://www.sqlite.org/fileformat.html and >> http://www.sqlite.org/fileformat2.ht

Re: [OSM-dev] osm2pgsql for 64-bit IDs

2011-05-24 Thread Frederik Ramm
Hi, On 05/24/11 17:05, Igor Brejc wrote: Not necessarily the same thing. Writing protobuf reader is still much easier than implementing something like http://www.sqlite.org/fileformat.html and http://www.sqlite.org/fileformat2.html If you say so - I thought that dealing with protobuf files wit

Re: [OSM-dev] osm2pgsql for 64-bit IDs

2011-05-24 Thread Peter Körner
Am 24.05.2011 17:05, schrieb Igor Brejc: Not necessarily the same thing. Writing protobuf reader is still much easier than implementing something like http://www.sqlite.org/fileformat.html and http://www.sqlite.org/fileformat2.html You don't need to *imlement* sqlite on your own, just as well a

Re: [OSM-dev] osm2pgsql for 64-bit IDs

2011-05-24 Thread Igor Brejc
Jukka, thanks for this info, it come very handy once I get back to working on spatialite stuff. Igor On Tue, May 24, 2011 at 1:00 PM, Jukka Rahkonen < jukka.rahko...@latuviitta.fi> wrote: > Igor Brejc kirjoitti: > > On Tue, May 24, 2011 at 11:39 AM, Frederik Ramm > > wrote: > > > >> Hi, > >> >

Re: [Potlatch-dev] Potlatch 2.1 and 2.x...

2011-05-24 Thread Andy Allan
> For those of you tracking the code: I've also started work on a new branch > ('refactor' at https://github.com/systemed/potlatch2) to form the basis of a > future version. This includes a huge bunch of under-the-hood changes with > two aims: > > - Allow multiple Maps to be displayed in one P2 ins

Re: [OSM-dev] osm2pgsql for 64-bit IDs

2011-05-24 Thread Igor Brejc
On Tue, May 24, 2011 at 1:23 PM, Frederik Ramm wrote: > Hi, > > > On 05/24/11 12:04, Igor Brejc wrote: > >> Yes, but is it really? It's a storage format, you need a 3rd party >> driver to read it >> > > Same for anything that uses protocol buffers, or for shapefiles, isn't it? Not necessarily t

Re: [OSM-dev] Mapnik fails to render tile (bug in mapnik?)

2011-05-24 Thread Frederik Ramm
Hi, On 05/24/11 13:31, Manuel Reimer wrote: I have a problem with the following view: http://www.openstreetmap.org/?mlat=49.993605&mlon=9.684956&zoom=18 OSM Inspector http://tools.geofabrik.de/osmi/?view=geometry&lat=49.993605&lon=9.684956&zoom=18 claims a "role mismatch" - you have tagged t

[OSM-dev] Mapnik fails to render tile (bug in mapnik?)

2011-05-24 Thread Manuel Reimer
Hello, I have a problem with the following view: http://www.openstreetmap.org/?mlat=49.993605&mlon=9.684956&zoom=18 The marked area is a farmyard. It initially was tagged as landuse=residential until I fixed this. Problem is, that it now is displayed in the right style in the upper part but sti

Re: [OSM-dev] osm2pgsql for 64-bit IDs

2011-05-24 Thread Frederik Ramm
Hi, On 05/24/11 12:04, Igor Brejc wrote: Yes, but is it really? It's a storage format, you need a 3rd party driver to read it Same for anything that uses protocol buffers, or for shapefiles, isn't it? and it's optimized for querying, not for storing high volume of data in an efficient manner

Re: [OSM-dev] osm2pgsql for 64-bit IDs

2011-05-24 Thread Jukka Rahkonen
Igor Brejc kirjoitti: > On Tue, May 24, 2011 at 11:39 AM, Frederik Ramm > wrote: > >> Hi, >> >> >> On 05/24/11 11:24, Igor Brejc wrote: >> >>> This could be a more serious issue. I guess in the history of GIS there >>> has never been such a large geo database as OSM is now becoming. Maybe >>> we (

Re: [OSM-dev] osm2pgsql for 64-bit IDs

2011-05-24 Thread Igor Brejc
On Tue, May 24, 2011 at 11:39 AM, Frederik Ramm wrote: > Hi, > > > On 05/24/11 11:24, Igor Brejc wrote: > >> This could be a more serious issue. I guess in the history of GIS there >> has never been such a large geo database as OSM is now becoming. Maybe >> we (as the OSM community) should take a

Re: [OSM-dev] osm2pgsql for 64-bit IDs

2011-05-24 Thread Jukka Rahkonen
Hi, Sorry for the previous accidentally sent mail. I was wondering why do we need 64-bit IDs after the osm2pgsql import but I would guess it is because of the nodes table is used for diff file updates. Generally speaking it might be better to let osm2pgsql to create a new serial column into osm_

Re: [OSM-dev] osm2pgsql for 64-bit IDs

2011-05-24 Thread Frederik Ramm
Hi, On 05/24/11 11:24, Igor Brejc wrote: This could be a more serious issue. I guess in the history of GIS there has never been such a large geo database as OSM is now becoming. Maybe we (as the OSM community) should take a proactive stance and propose a new version of shapefile format that coul

Re: [OSM-dev] osm2pgsql for 64-bit IDs

2011-05-24 Thread Igor Brejc
On Tue, May 24, 2011 at 10:52 AM, Jochen Topf wrote: > > One problem with 64bit IDs is simply that they need twice as much space. If > you > store a billion node IDs that might be the difference between needing 4GB > of > RAM or 8GB. So I think it is worth it trying to live with 32bit IDs as long

Re: [OSM-dev] osm2pgsql for 64-bit IDs

2011-05-24 Thread Jukka Rahkonen
Jochen Topf kirjoitti: > On Mon, May 23, 2011 at 04:18:05AM -0500, Scott Crosby wrote: >> On Sat, May 21, 2011 at 9:52 AM, Jochen Topf wrote: >> > >> > If we use unsigned ints we have some more time. Problematic would only >> be >> > a few cases where negative IDs are currently used (like in JOSM

Re: [OSM-dev] osm2pgsql for 64-bit IDs

2011-05-24 Thread Jochen Topf
On Mon, May 23, 2011 at 04:18:05AM -0500, Scott Crosby wrote: > On Sat, May 21, 2011 at 9:52 AM, Jochen Topf wrote: > > > > If we use unsigned ints we have some more time. Problematic would only be > > a few cases where negative IDs are currently used (like in JOSM for data > > thats not yet uploa