Hi Jin, Thanks for the progress report. You should only need the 'japan.osm.bz2' file. I would assume the the other osm.something.bz2 are sub-selections from that, so unless you use the --append flag to osm2pgsql each import will replace the other (and with --append you would likely get duplicate objects).
Dane On Nov 30, 2010, at 10:54 PM, Jin Kang wrote: > Hi Dane, > > Thanks for your input! Based on your suggestion, we installed virtual box > plus ubuntu combination and was able to made some progress. We now don't see > any error while rendering but still cannot see the following objects: > - building name > - river, forest, trees > - parking lot, bus mark > (arrow is being displayed now) > > FYI, we are getting data from following link: > http://downloads.cloudmade.com/asia/japan#downloads_breadcrumbs > > and using following 4 files. > * japan.osm.administrative.bz2 > * japan.osm.bz2 > * japan.osm.coastline.bz2 > * japan.osm.highway.bz2 > > Are there other files we need to use? In above link, there are 11 files as > below but we are not sure which is additionally needed? If need, we would > like to know how we can use those files. > japan.ai.zip (503.8M) > japan.img.zip (63.1M) > japan.navit.bin.zip (171M) > japan.osm.administrative.bz2 (72.1M) > japan.osm.bz2 (481.6M) > japan.osm.coastline.bz2 (46.4M) > japan.osm.highway.bz2 (43.1M) > japan.poi.gpx.zip (792.9K) > japan.poi.tomtom.zip (388.1K) > japan.poly (89.2K) > japan.shapefiles.zip (151.8M) > Thanks much in advance! > > -Jin > > > On Nov 28, 2010, at 1:55 AM, Dane Springmeyer wrote: > >> Jin, okay thanks very much for following through with this info. I'd >> recommend you cc yourself on the ticket I originally mentioned (see below, >> #581). That way we can track this general issue of generate_tiles on >> windows. >> >> Before giving up on windows entirely (I saw your other email requesting hw >> spec) you could test running Linux on your windows box. Virtualbox+ubuntu or >> the wubi installer are both good options if your windows machine has a >> decent amount of memory >=2mb. generate_tiles.py will run slower because of >> the filesystem overhead of the vm, but it should work for you to accomplish >> your task. >> >> However I should note I'm reponding based solely on your stated goal - to >> precache all of osm japan down to z16. Normally it's easier to just serve >> dynamically and simply cache on demand. Many of us here can advise on that >> approach as well. >> >> Dane >> >> --- \o/ --- >> Sent from my phone >> >> On Nov 26, 2010, at 12:33 AM, Jin Kang <[email protected]> wrote: >> >>> Hi Dane, >>> >>> Thanks for your comments and please refer to below. >>> >>>>> The problem was in PROJ_LIB's setting. After change the settings in >>>>> PROJ_LIB, we were able to run generate_tiles.py. We don't see any errors >>>>> but still we cannot see the street name or any character yet. >>>> >>>> Can you give more detail here? So the script continues to run with no >>>> errors and tiles are rendered just without anything but background color? >>>> >>> JIN: Yes, the script is running without errors. Just background color >>> without any text data. >>>>> >>>>> We checked the postgres' error log and found following errors. We kind of >>>>> suspect that this might be related to the fact that we are not getting >>>>> data described above. >>>>> >>>>> 2010-11-25 13:00:32 JST LOG: could not receive data from client: No >>>>> connection could be made because the target machine actively refused it. >>>>> 2010-11-25 13:00:32 JST LOG: unexpected EOF on client connection >>>> >>>> Yes, surely the culprit. Surprising this does not halt the script. Can you >>>> google this error a bit? I assume it has to do with Postgres<-->windows >>>> and perhaps there is a solution outside of Mapnik? >>>> >>>> Also did you get this same problem when running the multiprocess version >>>> of generate_tiles? >>> JIN: Yes, this happens with multiprocess as well. Regarding to database >>> connection, psql and pgAdmin can connect fine and firewall is turned off. >>> >>> Kind regards, >>> >>> -Jin >>> >>> >>> On Nov 26, 2010, at 1:42 PM, Dane Springmeyer wrote: >>> >>>> >>>> >>>> On Nov 25, 2010, at 12:18 AM, Jin Kang <[email protected]> wrote: >>>> >>>>> Hi Dane, >>>>> >>>>> Again thanks much for your comments! >>>>> >>>>> The problem was in PROJ_LIB's setting. After change the settings in >>>>> PROJ_LIB, we were able to run generate_tiles.py. We don't see any errors >>>>> but still we cannot see the street name or any character yet. >>>> >>>> Can you give more detail here? So the script continues to run with no >>>> errors and tiles are rendered just without anything but background color? >>>> >>>>> >>>>> We checked the postgres' error log and found following errors. We kind of >>>>> suspect that this might be related to the fact that we are not getting >>>>> data described above. >>>>> >>>>> 2010-11-25 13:00:32 JST LOG: could not receive data from client: No >>>>> connection could be made because the target machine actively refused it. >>>>> 2010-11-25 13:00:32 JST LOG: unexpected EOF on client connection >>>> >>>> Yes, surely the culprit. Surprising this does not halt the script. Can you >>>> google this error a bit? I assume it has to do with Postgres<-->windows >>>> and perhaps there is a solution outside of Mapnik? >>>> >>>> Also did you get this same problem when running the multiprocess version >>>> of generate_tiles? >>>> >>>>> >>>>> In osm.xm, we setup local database. Is this error familiar to you? >>>> >>>> No I've personally never seen it before but I doubt it is uncommon. >>>> >>>>> If you have any comments from your experience, it will helps us. >>>>> >>>>> Below are our S/W environment: >>>>> * PostgreSQL8.4.5/PostGIS1.5, >>>>> * Python2.6.6 >>>>> * mapnik0.7.1 >>>>> * proj.4.4.6 >>>>> >>>>> Regards, >>>>> >>>>> -Jin >>>>> >>>>> On Nov 25, 2010, at 5:13 AM, Dane Springmeyer wrote: >>>>> >>>>>> Yes, downloading the proj nad files and setting PROJ_LIB env setting is >>>>>> the way to solve that error, so all I can say is to double check your >>>>>> ENV settings. >>>>>> >>>>>> A workaround is to switch from using the epsg lookup syntax to +proj >>>>>> literal string in your stylesheet (and perhaps also in the >>>>>> generate_tiles.py code). This way you bypass the need for proj4 to >>>>>> lookup values in the epsg file. >>>>>> >>>>>> Dane >>>>>> >>>>>> --- \o/ --- >>>>>> Sent from my phone >>>>>> >>>>>> On Nov 24, 2010, at 4:29 AM, Jin Kang <[email protected]> wrote: >>>>>> >>>>>>> Hi Dane, >>>>>>> >>>>>>> Thank you very much for your help on this! As you suggested, we ran >>>>>>> shapeindex with single thread script (generate_tiles.py) but we got the >>>>>>> following error messages: >>>>>>> >>>>>>> proj_init_error:failed to initialize projection with:+init=epsg:4326 >>>>>>> World : 16 58199 25813 Empty Tile >>>>>>> propjr_oijnit_e_rirnoirt:_fearirolre:df atio iled to initniitailaliizze >>>>>>> e projec >>>>>>> >>>>>>> The above error might occurred since proj epsg file cannot be found so >>>>>>> we downloaded win32 binary from proj.4 and set the PATH and PROJ_LIB. >>>>>>> However, we are still getting same error message. The version of proj.4 >>>>>>> is 4.4.6. At this point, we are not sure how to resolve this. If you >>>>>>> have previous experience, please kindly let us know. >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> -Jin >>>>>>> >>>>>>> On Nov 23, 2010, at 2:05 AM, Dane Springmeyer wrote: >>>>>>> >>>>>>>> Hi Jin, >>>>>>>> >>>>>>>> Does the generate_tiles.py script visibly stop at some point? Is there >>>>>>>> any error visible in the terminal? >>>>>>>> >>>>>>>> My guess is that you running into a known windows+mapnik+postgtres >>>>>>>> problem. Basically that mapnik looses a proper database connection >>>>>>>> during tile generation a high zoom levels because rendering is fast >>>>>>>> and there is a higher likelyhood of postgres connections being >>>>>>>> clobbered. This has been known to happen on windows when using >>>>>>>> generate_tiles with threads >= 1 >>>>>>>> (http://trac.mapnik.org/ticket/581#comment:6). I see below that you >>>>>>>> are testing using the multiprocess script, but I would not be >>>>>>>> surprised if it is the same problem manifesting. Can you try with >>>>>>>> either 1 thread or 1 process? >>>>>>>> >>>>>>>> To narrow this down it would be good to know if you can render *any* >>>>>>>> tile at >= z16. One easy way to check with would be to install >>>>>>>> TileLite (http://bitbucket.org/springmeyer/tilelite/), then launch a >>>>>>>> server on localhost:8000 and try to view a z18 tile, like >>>>>>>> http://tile.openstreetmap.org/18/232909/103242.png at >>>>>>>> http://localhost:8000/18/232909/103242.png. >>>>>>>> >>>>>>>> One last thing to do is rebuilt your shape indexes: >>>>>>>> >>>>>>>> # on windows >>>>>>>> cd world_boundaries >>>>>>>> shapeindex processed_p >>>>>>>> shapeindex shoreline_300 >>>>>>>> shapeindex 10m-populated-places >>>>>>>> shapeindex 110m-admin-0-boundary-lines >>>>>>>> >>>>>>>> Dane >>>>>>>> >>>>>>>> On Nov 21, 2010, at 7:42 PM, Jin Kang wrote: >>>>>>>> >>>>>>>>> Hi everyone, >>>>>>>>> >>>>>>>>> My engineer is trying to do rendering up to level 16 but we kind of >>>>>>>>> stuck right now. We are pretty new in this application so if any of >>>>>>>>> you could help us on this, it would be great help to us. Please refer >>>>>>>>> to below for more detail info. >>>>>>>>> >>>>>>>>> ------------------------------------------------------------------------- >>>>>>>>> We would like to do rendering OSM up to zoom level 16 using mapnik. >>>>>>>>> Right now, we are just focusing on Japanese map only which you can >>>>>>>>> find at http://www.openstreetmap.jp/. >>>>>>>>> >>>>>>>>> We also referred to following link but didn't get the expected result >>>>>>>>> so far. >>>>>>>>> http://wiki.openstreetmap.org/wiki/Mapnik >>>>>>>>> http://wiki.openstreetmap.org/wiki/DE:Mapnik >>>>>>>>> >>>>>>>>> The problem is that we can only render up to zoom level 10, not 16 >>>>>>>>> and at zoom level 11, we basically cannot see anything. Even at zoom >>>>>>>>> level 10, we cannot see any data such as street name, country name or >>>>>>>>> anything. >>>>>>>>> >>>>>>>>> Regarding OSM, we imported from following link. >>>>>>>>> http://downloads.cloudmade.com/asia/japan#downloads_breadcrumbs >>>>>>>>> ＊japan.osm.bz2 >>>>>>>>> ＊japan.osm.coastline.bz2 >>>>>>>>> ＊japan.osm.highway.bz2 >>>>>>>>> >>>>>>>>> We followed following steps: >>>>>>>>> 1. Install PostgreSQL8.4.5/PostGIS1.5, Python2.6.6, mapnik0.7.1 >>>>>>>>> >>>>>>>>> 2. Create database >>>>>>>>> $ createdb -U postgres -E UTF8 -O mapnik gis >>>>>>>>> >>>>>>>>> 3. Run sql >>>>>>>>> $ psql -U postgres -d gis -f "C:\Program >>>>>>>>> Files\PostgreSQL\8.4\share\contrib\postgis-1.5\postgis.sql" >>>>>>>>> $ psql -U postgres -d gis -f "C:\Program >>>>>>>>> Files\PostgreSQL\8.4\share\contrib\postgis-1.5\spatial_ref_sys.sql" >>>>>>>>> $ psql -U postgres -d gis -f "C:\Program >>>>>>>>> Files\PostgreSQL\8.4\share\contrib\_int.sql" >>>>>>>>> $ psql -U postgres -d gis -f "C:\Program >>>>>>>>> Files\PostgreSQL\8.4\share\contrib\hstore.sql" >>>>>>>>> >>>>>>>>> 4. Run osm2pgsql >>>>>>>>> $ osm2pgsql -c -s -d gis -U postgres -S default.style japan.osm.bz2 >>>>>>>>> $ osm2pgsql -c -s -d gis -U postgres -S default.style >>>>>>>>> japan.osm.highway.bz2 >>>>>>>>> $ osm2pgsql -c -s -d gis -U postgres -S default.style >>>>>>>>> japan.osm.coastline.bz2 >>>>>>>>> $ osm2pgsql -c -s -d gis -U postgres -S default.style >>>>>>>>> japan.osm.administrative.bz2 >>>>>>>>> >>>>>>>>> 5. world_boundaries(in mac) >>>>>>>>> $ tar xzf world_boundaries-spherical.tgz >>>>>>>>> $ tar xjf processed_p.tar.bz2 -C world_boundaries >>>>>>>>> $ tar xjf shoreline_300.tar.bz2 -C world_boundaries >>>>>>>>> $ unzip 10m-populated-places.zip -d world_boundaries >>>>>>>>> $ unzip 110m-admin-0-boundary-lines.zip -d world_boundaries >>>>>>>>> >>>>>>>>> 6. Check out mapnik repository >>>>>>>>> URL: http://svn.openstreetmap.org/applications/rendering/mapnik >>>>>>>>> >>>>>>>>> 7. Create mapnik setup file >>>>>>>>> $ generate_xml.py osm.xml out.xml --host localhost --user postgres >>>>>>>>> --dbname gis --symbols ./symbols/ --world_boundaries >>>>>>>>> ./world_boundaries/ >>>>>>>>> --port 5432 --password '' >>>>>>>>> >>>>>>>>> 8. Using generate_tiles_multiprocess.py, do rendering >>>>>>>>> $ set HOME=C:\mapnik-0.7.1\svn\ >>>>>>>>> $ set MAPNIK_MAP_FILE=C:\mapnik-0.7.1\svn\out.xml >>>>>>>>> $ set MAPNIK_TILE_DIR=C:\mapnik-0.7.1\svn\tiles\ >>>>>>>>> $ generate_tiles_multiprocess.py >>>>>>>>> >>>>>>>>> The portion we changed to for Japanese map >>>>>>>>> # zoom level=16- describe Japan >>>>>>>>> bbox = (112.5,22.0,157.5,55.7) >>>>>>>>> render_tiles(bbox, mapfile, tile_dir, 16, 16 , "Japan") >>>>>>>>> # zoom level=10- describe Japan >>>>>>>>> bbox = (112.5,22.0,157.5,55.7) >>>>>>>>> render_tiles(bbox, mapfile, tile_dir, 10, 10 , "Japan") >>>>>>>>> >>>>>>>>> Hardware spec: >>>>>>>>> PC Spec >>>>>>>>> CPU：Core2 6700 2.66G >>>>>>>>> RAM：3G >>>>>>>>> Hard Disk：256G >>>>>>>>> OS：Windows XP SP3 >>>>>>>>> _______________________________________________ >>>>>>>>> Mapnik-users mailing list >>>>>>>>> [email protected] >>>>>>>>> https://lists.berlios.de/mailman/listinfo/mapnik-users >>>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Mapnik-users mailing list >>>>>>> [email protected] >>>>>>> https://lists.berlios.de/mailman/listinfo/mapnik-users >>>>> >>> >
_______________________________________________ Mapnik-users mailing list [email protected] https://lists.berlios.de/mailman/listinfo/mapnik-users
