Re: [OSM-talk] rendering with mapnik, what do I need?
Am Samstag, 1. März 2008 18:27:42 schrieb Jo: > Gervase Markham schreef: > 2. What do I need? PostgreSQL? Python? What else? > >>> > >>> See http://wiki.openstreetmap.org/index.php/Mapnik > >> > >> It's rather short. Is it really that simple...? > > > > I'm surprised by that reaction... I just read the Mapnik page and > > thought "that's flipping complicated". > > > > Mapnik itself, judging by http://www.mapnik.org/, seems to regard the > > whole PosgreSQL thing as optional. Is there any way of getting OSM data > > into a format Mapnik will render without having to set up a database? > > I'm familiar with setting up Postgres and importing a dump file into it. > So that didn't put me off. I must admit though that I didn't try it yet. > It might prove harder than I can foresee right now. Setting up the mapnik database should be as easy as to run the mapnik-osm-updater.sh script. This script will set everything up including creation of users, database and downloading planet File) -- Joerg -- Jörg (Germany, Munich) http://www.ostertag.name/ irc://irc.oftc.net/#osm Tel.: +49 89 420950304 Skype: JoergOstertag ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] rendering with mapnik, what do I need?
Gervase Markham schreef: 2. What do I need? PostgreSQL? Python? What else? >>> See http://wiki.openstreetmap.org/index.php/Mapnik >>> >> It's rather short. Is it really that simple...? >> > > I'm surprised by that reaction... I just read the Mapnik page and > thought "that's flipping complicated". > > Mapnik itself, judging by http://www.mapnik.org/, seems to regard the > whole PosgreSQL thing as optional. Is there any way of getting OSM data > into a format Mapnik will render without having to set up a database? > I'm familiar with setting up Postgres and importing a dump file into it. So that didn't put me off. I must admit though that I didn't try it yet. It might prove harder than I can foresee right now. Polyglot ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] rendering with mapnik, what do I need?
On Fri, Feb 29, 2008 at 11:21 PM, Gervase Markham <[EMAIL PROTECTED]> wrote: > Mapnik itself, judging by http://www.mapnik.org/, seems to regard the > whole PosgreSQL thing as optional. Is there any way of getting OSM data > into a format Mapnik will render without having to set up a database? Well, mapnik treats the postgis input more like a glorified shapefile. You could in theory dump the OSM data into shapefiles and render from there. Fact is though, running it from the database is a lot faster because Postgres is very smart about optimising queries. The coastline checker produces output in shapefile format which you can use in mapnik directly, however dumping it into a database improves the performance. Have a nice day, -- Martijn van Oosterhout <[EMAIL PROTECTED]> http://svana.org/kleptog/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] rendering with mapnik, what do I need?
On Fri, 2008-02-29 at 22:21 +, Gervase Markham wrote: > >>> 2. What do I need? PostgreSQL? Python? What else? > >> See http://wiki.openstreetmap.org/index.php/Mapnik > > It's rather short. Is it really that simple...? > > I'm surprised by that reaction... I just read the Mapnik page and > thought "that's flipping complicated". > > Mapnik itself, judging by http://www.mapnik.org/, seems to regard the > whole PosgreSQL thing as optional. Is there any way of getting OSM data > into a format Mapnik will render without having to set up a database? You could try: https://lists.berlios.de/pipermail/mapnik-devel/2008-February/000514.html ... but the Postgres based approach has been used by the main OSM site for over a year so it it pretty well tested and understood. The link above is new code which was only announced 3 days ago. Postgres is quite easy to work with once you've gone through the pain of setting it up the first time. For example, to import the new data each week you just run: osm2pgsql planet-latest.osm.bz2 Jon ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] rendering with mapnik, what do I need?
On Fri, Feb 29, 2008 at 10:21:04PM +, Gervase Markham wrote: > >>> 2. What do I need? PostgreSQL? Python? What else? > >> See http://wiki.openstreetmap.org/index.php/Mapnik > > It's rather short. Is it really that simple...? > > I'm surprised by that reaction... I just read the Mapnik page and > thought "that's flipping complicated". > > Mapnik itself, judging by http://www.mapnik.org/, seems to regard the > whole PosgreSQL thing as optional. Is there any way of getting OSM data > into a format Mapnik will render without having to set up a database? There is active ongoing development to support rendering with mapnik directly from a .osm file, I believe. Regards, -- Christopher Schmidt MetaCarta ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] rendering with mapnik, what do I need?
>>> 2. What do I need? PostgreSQL? Python? What else? >> See http://wiki.openstreetmap.org/index.php/Mapnik > It's rather short. Is it really that simple...? I'm surprised by that reaction... I just read the Mapnik page and thought "that's flipping complicated". Mapnik itself, judging by http://www.mapnik.org/, seems to regard the whole PosgreSQL thing as optional. Is there any way of getting OSM data into a format Mapnik will render without having to set up a database? Gerv ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] rendering with mapnik, what do I need?
Hello, On Fri, 22 Feb 2008 10:01:39 +0100, Jo <[EMAIL PROTECTED]> wrote: > Say I would like to set up a server myself. I want to: > > * show highways more with the colours used on Michelin maps > * show a bicycle map as three overlays (transparent, with a possibility > to switch them on and off) > * show bus routes for each bus separately as overlays (transparent layers) I'm *very* interested in doing something similar, particularly the bus routes as transparent overlays. If you make any progress please keep me in the loop! It will be for this web site: www.oneplanetsutton.org Kind regards, Tom ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] rendering with mapnik, what do I need?
On Fri, Feb 22, 2008 at 2:29 PM, Andy Allan <[EMAIL PROTECTED]> wrote: > > On Fri, Feb 22, 2008 at 2:18 PM, Dave Stubbs <[EMAIL PROTECTED]> wrote: > > On Fri, Feb 22, 2008 at 1:50 PM, Jo <[EMAIL PROTECTED]> wrote: > > > Do the transparent layers take the same amount of time to render as the > > > base layer for a given area? > > > > > > > > > This will depend entirely on what you put on them, and how you > > organise that data in the database. > > Assuming your transparent layer only has a few lines (ie: bus routes) > > and you keep this data in a separate table, then they'll render > > faster, probably a lot faster. > > ...than normal background tiles. I don't think Dave meant to imply > that it's quicker to render "background tiles" + "overlay tiles" than > it is to render one set of "background + overlay" tiles. Or in other > words, every line that gets drawn takes a certain amount of time. It > doesn't matter what the background colour is. > > If you only want to draw some overlays for the standard OSM tiles then > you're quicker just drawing the overlays. > I was answering the question: is mapnik rendering speed limited by pixel area, or by data density? To which the answer is data density for the base tiles. I'm still trying to work out what Andy thinks I said, which probably says more about my brain than his. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] rendering with mapnik, what do I need?
On Fri, Feb 22, 2008 at 2:18 PM, Dave Stubbs <[EMAIL PROTECTED]> wrote: > On Fri, Feb 22, 2008 at 1:50 PM, Jo <[EMAIL PROTECTED]> wrote: > > Do the transparent layers take the same amount of time to render as the > > base layer for a given area? > > > > > This will depend entirely on what you put on them, and how you > organise that data in the database. > Assuming your transparent layer only has a few lines (ie: bus routes) > and you keep this data in a separate table, then they'll render > faster, probably a lot faster. ...than normal background tiles. I don't think Dave meant to imply that it's quicker to render "background tiles" + "overlay tiles" than it is to render one set of "background + overlay" tiles. Or in other words, every line that gets drawn takes a certain amount of time. It doesn't matter what the background colour is. If you only want to draw some overlays for the standard OSM tiles then you're quicker just drawing the overlays. Cheers, Andy ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] rendering with mapnik, what do I need?
On Fri, Feb 22, 2008 at 2:09 PM, Chris Jones <[EMAIL PROTECTED]> wrote: > On Fri, 22 Feb 2008, Jo wrote: > > > >>> 2. What do I need? PostgreSQL? Python? What else? > >> > >> See http://wiki.openstreetmap.org/index.php/Mapnik > > It's rather short. Is it really that simple...? How often is the planet > file > > generated? Say I would like to have daily updates instead of weekly. Is > that > > possible? I guess I should just try it... > > You can complecate it more if you want to render things on demand rather > than prerender everything, but thats basicly all thats needed produce a > bunch of tiles. > > There are daily and even hourly 'diffs', i dont know if osm2pgsql or any > other tool can update the mapnik database useing them though. > Not at the moment, but if you're only working on a smallish excerpt of the planet it would probably be feasible to apply the daily diff and reimport the data to postgres in not-much-time. This doesn't really work for the whole planet where importing alone seems to be taking about 4 hours... this would probably go faster if I had faster disks, and if osm2pgsql could take advantage of all my CPUs - but that's non-trivial. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] rendering with mapnik, what do I need?
On Fri, Feb 22, 2008 at 2:09 PM, Chris Jones <[EMAIL PROTECTED]> wrote: > On Fri, 22 Feb 2008, Jo wrote: > > > >>> 2. What do I need? PostgreSQL? Python? What else? > >> > >> See http://wiki.openstreetmap.org/index.php/Mapnik > > It's rather short. Is it really that simple...? How often is the planet > file > > generated? Say I would like to have daily updates instead of weekly. Is > that > > possible? I guess I should just try it... > > You can complecate it more if you want to render things on demand rather > than prerender everything, but thats basicly all thats needed produce a > bunch of tiles. See also http://wiki.openstreetmap.org/index.php/Deploying_your_own_Slippy_Map > There are daily and even hourly 'diffs', i dont know if osm2pgsql or any > other tool can update the mapnik database useing them though. I'm not sure, but you can always apply the diffs to the planet file itself and re-import. Cheers, Andy ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] rendering with mapnik, what do I need?
On Fri, Feb 22, 2008 at 1:50 PM, Jo <[EMAIL PROTECTED]> wrote: > Do the transparent layers take the same amount of time to render as the > base layer for a given area? > This will depend entirely on what you put on them, and how you organise that data in the database. Assuming your transparent layer only has a few lines (ie: bus routes) and you keep this data in a separate table, then they'll render faster, probably a lot faster. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] rendering with mapnik, what do I need?
On Fri, 22 Feb 2008, Jo wrote: >>> 2. What do I need? PostgreSQL? Python? What else? >> >> See http://wiki.openstreetmap.org/index.php/Mapnik > It's rather short. Is it really that simple...? How often is the planet file > generated? Say I would like to have daily updates instead of weekly. Is that > possible? I guess I should just try it... You can complecate it more if you want to render things on demand rather than prerender everything, but thats basicly all thats needed produce a bunch of tiles. There are daily and even hourly 'diffs', i dont know if osm2pgsql or any other tool can update the mapnik database useing them though. -- Chris Jones, SUCS Admin http://sucs.org ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] rendering with mapnik, what do I need?
Do the transparent layers take the same amount of time to render as the base layer for a given area? Polyglot ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] rendering with mapnik, what do I need?
Hi Chris, Thanks for your answer. Chris Jones wrote: > On Fri, 22 Feb 2008, Jo wrote: > >> Say I would like to set up a server myself. I want to: >> >> * show highways more with the colours used on Michelin maps >> * show a bicycle map as three overlays (transparent, with a possibility >> to switch them on and off) >> * show bus routes for each bus separately as overlays (transparent >> layers) >> >> 1. Is this possible? > > Yes Great. > >> 2. What do I need? PostgreSQL? Python? What else? > > See http://wiki.openstreetmap.org/index.php/Mapnik It's rather short. Is it really that simple...? How often is the planet file generated? Say I would like to have daily updates instead of weekly. Is that possible? I guess I should just try it... > >> 3. What kind of server would be good for this? Would an AMD64 with one >> core be able to do this? Do I need more cores? Does 64 bit processing >> help? Or would I be just as well off with a Core2Duo or even a >> Pentium 4? > > Any recent processor is fine, obviously the faster the core(s) the > more tiles you can render in a given time. So an AMD64 would be OK then. > >> 4. For the disks I was considering to set up a RAID1 mirror with 2 disks >> of 750 GB. I guess that would be sufficient? > > How much storage you need depends on how much of the world you would > like to render and to what zoom level. > > I render the whole UK to level 5 and just Wales to zoom 16, this uses > a mere 450MB > > In my experience each zoom level is roughly 2.5x bigger (in terms of > disk space) than the previous. > > Unless your planning to serve the whole world, to hundreds of clients > at a time (which your connection info below suggests your not) RAID1 > is massive overkill! The regularly requested tiles will get cached in > memory anyway further reducing the disk speed requirements. I'm not sure if I want the whole world. I do want to render Belgium and probably a good part of mainland Europe. The server would also be used for my own purposes. Firewall, OpenLDAP, Postfix, HTTP, Proxy server, Asterisk, maybe MythTV, LTSP, Nagios. I simply feel more comfortable with a RAID1 setup. I wouldn't mind it uses spare cycles to render a map. > > I serve (pre-rendered) tiles from a NSLU2* running Debian with a USB > stick for storage. This setup quite happily serves ~150 tiles per second. > > *http://en.wikipedia.org/wiki/NSLU2 > >> I would like to do this from 'home' with a connection that has a fixed >> IP address and 1 or 2 Mbps upload. I'll probably be shaping the traffic >> to limit it and make my own internet connection still usable. > > With a 2 Mbps upload you should be able to serve about 40 tiles per second That should be enough. I'll probably shape it down to 700kbps then. Polyglot ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] rendering with mapnik, what do I need?
On Fri, 22 Feb 2008, Jo wrote: > Say I would like to set up a server myself. I want to: > > * show highways more with the colours used on Michelin maps > * show a bicycle map as three overlays (transparent, with a possibility > to switch them on and off) > * show bus routes for each bus separately as overlays (transparent layers) > > 1. Is this possible? Yes > 2. What do I need? PostgreSQL? Python? What else? See http://wiki.openstreetmap.org/index.php/Mapnik > 3. What kind of server would be good for this? Would an AMD64 with one > core be able to do this? Do I need more cores? Does 64 bit processing > help? Or would I be just as well off with a Core2Duo or even a Pentium 4? Any recent processor is fine, obviously the faster the core(s) the more tiles you can render in a given time. > 4. For the disks I was considering to set up a RAID1 mirror with 2 disks > of 750 GB. I guess that would be sufficient? How much storage you need depends on how much of the world you would like to render and to what zoom level. I render the whole UK to level 5 and just Wales to zoom 16, this uses a mere 450MB In my experience each zoom level is roughly 2.5x bigger (in terms of disk space) than the previous. Unless your planning to serve the whole world, to hundreds of clients at a time (which your connection info below suggests your not) RAID1 is massive overkill! The regularly requested tiles will get cached in memory anyway further reducing the disk speed requirements. I serve (pre-rendered) tiles from a NSLU2* running Debian with a USB stick for storage. This setup quite happily serves ~150 tiles per second. *http://en.wikipedia.org/wiki/NSLU2 > I would like to do this from 'home' with a connection that has a fixed > IP address and 1 or 2 Mbps upload. I'll probably be shaping the traffic > to limit it and make my own internet connection still usable. With a 2 Mbps upload you should be able to serve about 40 tiles per second -- Chris Jones, SUCS Admin http://sucs.org ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
