Re: [Talk-us] Great Lakes missing after osmosis update (diffs installation)
Thanks for responding Kevin, Actually I have the issue after daily updating the whole planet. But effectively, I don’t have that issue when I use only Ontario province or all Canada with geofabric daily updates… Only with the whole planet daily update…. (same issue with hourly and minutely updates). I am using a 32-cores server with 62 GB RAM and 1.4TB SSD. ALL: Am I the only one downloading the whole planet and trying to daily update it successfully? Please share your experience if you update the whole planet daily, hourly or minutely… Thanks, Michel From: kevin.b.ke...@gmail.com [mailto:kevin.b.ke...@gmail.com] On Behalf Of Kevin Kenny Sent: October 3, 2017 13:49 To: Pilon, Michel (SSC/SPC) Cc: talk-us@openstreetmap.org Subject: Re: [Talk-us] Great Lakes missing after osmosis update (diffs installation) On Tue, Oct 3, 2017 at 1:31 PM, Pilon, Michel (SSC/SPC) mailto:michel.pi...@canada.ca>> wrote: Have you gotten a chance to validate the coastlines of the Great Lakes during the last weekend? You’re the only one who responded to my “help” request…. Oops, sorry, I thought I'd sent a reply. I found one self-intersection on Lake Michigan and fixed it. Aside from that, there didn't seem to be any problems, and the Great Lakes render all right on my machine. (Even before the fix, they were rendering all right.) I'm not a Mapnik maintainer, nor a really sophisticated user, so I'm not certain what might be amiss. Could it be that you had a failed import of diffs that broke something? (In which case, there might be no alternative to rerunning osm2pgsql from a planet export.) Does your app need up-to-the-minute global data? I usually run with just North America from geofabrik, and only use geofabrik's daily diffs. I started out trying to maintain a mirror with minutely diffs, and it was kind of a nightmare. If anything went wrong even momentarily, it struggled to catch up and eventually failed. This was with a quadcore, 32 GB main memory, 0.5 TB SSD / 4 TB RAID0+1. It may have been bandwidth limited, but I think it was more that you need a bigger cache than I could give it. That's a big database to try to synchronize remotely. Otherwise, I'm kind of out of ideas. Maybe others who hear this message can help more. ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Great Lakes missing after osmosis update (diffs installation)
On Tue, Oct 3, 2017 at 1:31 PM, Pilon, Michel (SSC/SPC) < michel.pi...@canada.ca> wrote: > Have you gotten a chance to validate the coastlines of the Great Lakes > during the last weekend? > > You’re the only one who responded to my “help” request…. > Oops, sorry, I thought I'd sent a reply. I found one self-intersection on Lake Michigan and fixed it. Aside from that, there didn't seem to be any problems, and the Great Lakes render all right on my machine. (Even before the fix, they were rendering all right.) I'm not a Mapnik maintainer, nor a really sophisticated user, so I'm not certain what might be amiss. Could it be that you had a failed import of diffs that broke something? (In which case, there might be no alternative to rerunning osm2pgsql from a planet export.) Does your app need up-to-the-minute global data? I usually run with just North America from geofabrik, and only use geofabrik's daily diffs. I started out trying to maintain a mirror with minutely diffs, and it was kind of a nightmare. If anything went wrong even momentarily, it struggled to catch up and eventually failed. This was with a quadcore, 32 GB main memory, 0.5 TB SSD / 4 TB RAID0+1. It may have been bandwidth limited, but I think it was more that you need a bigger cache than I could give it. That's a big database to try to synchronize remotely. Otherwise, I'm kind of out of ideas. Maybe others who hear this message can help more. ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Great Lakes missing after osmosis update (diffs installation)
Hello Kevin, Have you gotten a chance to validate the coastlines of the Great Lakes during the last weekend? You’re the only one who responded to my “help” request…. Thanks, Michel From: kevin.b.ke...@gmail.com [mailto:kevin.b.ke...@gmail.com] On Behalf Of Kevin Kenny Sent: September 28, 2017 16:37 To: Pilon, Michel (SSC/SPC) Cc: talk-us@openstreetmap.org Subject: Re: [Talk-us] Great Lakes missing after osmosis update (diffs installation) On Thu, Sep 28, 2017 at 1:23 PM, Pilon, Michel (SSC/SPC) mailto:michel.pi...@canada.ca>> wrote: Hello all, I am in the process of putting in place a local OSM server. Last Thursday September 21st I populated my PostGIS database using the following command: osm2pgsql --create --slim --flat-nodes /data/osm/pgload/flat_nodes.bin \ -C 27000 --number-processes 8 --hstore \ --style /data/osm/openstreetmap-carto-3.2.0/openstreetmap-carto.style \ --multi-geometry /data/osm/new_planet-latest.osm.pbf Once done I verified the slippymap and I can see correctly all the Canadian/US Great Lakes. So I applied the daily updates on the data using osmosis (daily diff). Once completed, everything looks good EXCEPT that there is no more water on Great Lakes except Lac Érié and Ontario for ALL ZOOM LEVELS. Superior, Michigan and Huron Lakes just disappeared! For the update process, I am using /usr/bin/openstreetmap-tiles-update-expire with the followng options for osm2pgsql command: OSM2PGSQL_OPTIONS="--number-processes 8 --hstore -C 27000 --style /data/osm/openstreetmap-carto-3.2.0/openstreetmap-carto.style --multi-geometry" I am using mapnik with openstreetmap-carto stylesheets. I am really stuck and need help as I tried it 3 times, always with the result of Lakes disappearing for all zoom levels… ☹ Any ideas??? Thanks in advanced for your very appreciated help…. Someone probably broke the coastline again. It's easy to goof when editing the lake shore. I don't have time to look into it right now, I can try on the weekend if someone else doesn't fix it first. (And, I confess, I've broken the lake shore of Lake Ontario at least once. I fixed it promptly once I realized the problem.) ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Great Lakes missing after osmosis update (diffs installation)
On Thu, Sep 28, 2017 at 1:23 PM, Pilon, Michel (SSC/SPC) < michel.pi...@canada.ca> wrote: > Hello all, > > > > I am in the process of putting in place a local OSM server. > > > > Last Thursday September 21st I populated my PostGIS database using the > following command: > > > > *osm2pgsql --create --slim --flat-nodes /data/osm/pgload/flat_nodes.bin \* > > * -C 27000 --number-processes 8 --hstore \* > > * --style > /data/osm/openstreetmap-carto-3.2.0/openstreetmap-carto.style \* > > * --multi-geometry /data/osm/new_planet-latest.osm.pbf* > > > > Once done I verified the slippymap and I can see correctly all the > Canadian/US Great Lakes. > > > > So I applied the daily updates on the data using osmosis (daily diff). > Once completed, everything looks good EXCEPT that there is no more water on > Great Lakes except Lac Érié and Ontario for *ALL ZOOM LEVELS*. Superior, > Michigan and Huron Lakes just disappeared! > > > > For the update process, I am using /usr/bin/openstreetmap-tiles-update-expire > with the followng options for osm2pgsql command: > >*OSM2PGSQL_OPTIONS="--number-processes 8 --hstore -C 27000 > --style /data/osm/openstreetmap-carto-3.2.0/openstreetmap-carto.style > --multi-geometry"* > > > > I am using mapnik with openstreetmap-carto stylesheets. > > > > I am really stuck and need help as I tried it 3 times, always with the > result of Lakes disappearing for all zoom levels… L > > > > Any ideas??? > > > > Thanks in advanced for your very appreciated help…. > > > > Someone probably broke the coastline again. It's easy to goof when editing the lake shore. I don't have time to look into it right now, I can try on the weekend if someone else doesn't fix it first. (And, I confess, I've broken the lake shore of Lake Ontario at least once. I fixed it promptly once I realized the problem.) ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
[Talk-us] Great Lakes missing after osmosis update (diffs installation)
Hello all, I am in the process of putting in place a local OSM server. Last Thursday September 21st I populated my PostGIS database using the following command: osm2pgsql --create --slim --flat-nodes /data/osm/pgload/flat_nodes.bin \ -C 27000 --number-processes 8 --hstore \ --style /data/osm/openstreetmap-carto-3.2.0/openstreetmap-carto.style \ --multi-geometry /data/osm/new_planet-latest.osm.pbf Once done I verified the slippymap and I can see correctly all the Canadian/US Great Lakes. So I applied the daily updates on the data using osmosis (daily diff). Once completed, everything looks good EXCEPT that there is no more water on Great Lakes except Lac Érié and Ontario for ALL ZOOM LEVELS. Superior, Michigan and Huron Lakes just disappeared! For the update process, I am using /usr/bin/openstreetmap-tiles-update-expire with the followng options for osm2pgsql command: OSM2PGSQL_OPTIONS="--number-processes 8 --hstore -C 27000 --style /data/osm/openstreetmap-carto-3.2.0/openstreetmap-carto.style --multi-geometry" I am using mapnik with openstreetmap-carto stylesheets. I am really stuck and need help as I tried it 3 times, always with the result of Lakes disappearing for all zoom levels... :( Any ideas??? Thanks in advanced for your very appreciated help Michel -- Michel Pilon Conseiller technique / Technical Advisor Courriel | Email : michel.pi...@canada.ca Téléphone | Telephone : (819) 564-5600 ext. 227 Télécopieur | Fax: (819) 564-5698 50 Place de la Cité, suite 212, C.P. 162, Sherbrooke (Québec) J1H 4G9 Échange de données, solutions d'E/S E-Science | E-Science Input/Output Solutions Data Superinformatique | Super Computing Gestion des services et centres de données | Service Management and Data Centres Services partagés Canada | Shared Services Canada Gouvernement du Canada | Government of Canada ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
