Re: [Talk-us] TopOSM

[Lars Ahlzen]

Hi Kevin,

On 05/22/2016 11:26 PM, Kevin Kenny wrote:
Alas, I'm not going to SOTM, but put me down as someone who's 
interested in the project, with some 'skin in the game' already.


As I already posted privately to Clifford:

I picked up TopOSM's code for my own purposes and added quite a few 
twists of my own. I use the result as a basemap for several of my own 
projects. You can see what it looks like at 
https://kbk.is-a-geek.net/catskills/test3.html.


You've done some great work on the topo maps! I've been following it for 
quite some time. It would fantastic if you're interested in closer 
collaboration.



Clearly, I am NOT tooled up to serve it up on a large scale.

It depends on a good many publicly-available data layers with 
ODBL-incompatible terms. Life is full of tradeoffs. I think that a 
'sanitized' version with only ODBL and US Government data wouldn't be 
too difficult to put together.


I'm more than willing to share the code, but it would be a bit of a 
nightmare to set up. I think that the best approach would be to share 
it with a willing apprentice (if you will) in pieces, reworking as we 
go to make sure that each shared piece runs for more than just me and 
the setup is better documented than it is now. I'm willing to put in 
the effort to make such a project succeed, but would find it immensely 
difficult without a guinea pig to try stuff out and provide ongoing 
feedback.


Sounds like a good plan. I guess we should take further details off 
list, but I think this is well worth giving a try.


Significant public layers that I include that I believe TopOSM does 
not include:


(1) NLCD [...]

(2) A fair number of FCODES from NHD [...]

(3) The USFWS National Wetlands Inventory. [...]


I did (1) and (2) in TopOSM2 (and some related projects I've been 
working on since) but, of course, that was never finished so I guess 
that's moot. Either way, it's probably good that we've been heading in 
the same direction.


- Lars


___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM

[Kevin Kenny]

On 05/22/2016 02:39 PM, Lars Ahlzen wrote:

On 05/21/2016 01:54 PM, Clifford Snow wrote:
TopOSM looks like a good candidate for a hack session at SOTM-US. Let 
me know if you are interested so I can find a room for people to meet 
on July 25th.


That's not a bad idea. Sounds like there's some interest in the OSM 
community, and I'd be happy to take a look at it again. I'd say go ahead!


Alas, I'm not going to SOTM, but put me down as someone who's interested 
in the project, with some 'skin in the game' already.


As I already posted privately to Clifford:

I picked up TopOSM's code for my own purposes and added quite a few 
twists of my own. I use the result as a basemap for several of my own 
projects. You can see what it looks like at 
https://kbk.is-a-geek.net/catskills/test3.html.


The 'catskills' in the name is something of a misnomer - it started as a 
Catskill Mountain project, but got out of hand and now covers most of 
the Eastern Seaboard. The 'test' in the name reflects that the page in 
question was intended largely as a demonstration viewer. I mostly use 
the tiles in apps on my phone.


At the moment, it's several months out of date. I've generally brought 
it fully up to date only a few times a year, because I've never troubled 
to get a pipeline going for automatic rendering with a tool such as 
tilestache. Given the limited area that it supports, I do all right 
serving static tiles. It generally takes overnight to produce them all, 
and warms my home office quite nicely when it's running. Clearly, I am 
NOT tooled up to serve it up on a large scale.


It depends on a good many publicly-available data layers with 
ODBL-incompatible terms. Life is full of tradeoffs. I think that a 
'sanitized' version with only ODBL and US Government data wouldn't be 
too difficult to put together.


I'm more than willing to share the code, but it would be a bit of a 
nightmare to set up. I think that the best approach would be to share it 
with a willing apprentice (if you will) in pieces, reworking as we go to 
make sure that each shared piece runs for more than just me and the 
setup is better documented than it is now. I'm willing to put in the 
effort to make such a project succeed, but would find it immensely 
difficult without a guinea pig to try stuff out and provide ongoing 
feedback.


Significant public layers that I include that I believe TopOSM does not 
include:


(1) NLCD, the National Land Cover Database. This provides the base 
colour, atop which is added hillshading. For my personal map, this is 
not something I'm willing to live without. It's important to me, when 
planning an off-trail trip, to know when to expect coniferous forest 
(near me, this is generally dense balsam-and-spruce that slows travel to 
a crawl) and when to expect deciduous forest (which, by contrast, is 
relatively open). It also pretty much obviates the need to use 
NaturalEarth's 1:10million populated areas shading, which is at an 
inappropriately coarse scale for most of our use of it.


In order to do this, I had to replace styles describing land use/land 
cover/cadastre to use shading on the boundaries rather than area fills. 
(I think the appearance in the end is nicer in any case.) I confess to 
colour-blindness and would welcome a more attractive colour scheme.


(2) A fair number of FCODES from NHD (National Hydrographic Dataset) 
that TopOSM ignores. Ignoring these FCODEs loses, for instance, the 
lower Hudson River on TopOSM.


(3) The USFWS National Wetlands Inventory. Most of the marshlands that 
are shown on my map come from there.


There are also a few state-level data sets describing foot trails and 
public land boundaries in my map that I am NOT comfortable including in 
TopOSM without consulting with the issuing agencies. Some of the 
agencies are not amenable to allowing redistribution of their data, even 
when the law requires them to make the data available to the public. 
Most of those are shown on the map in magenta, and serve me as a "things 
to do" list when I'm thinking up ideas for a short hiking trip.





--
73 de ke9tv/2, Kevin


___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM

[Lars Ahlzen]

On 05/21/2016 01:54 PM, Clifford Snow wrote:
TopOSM looks like a good candidate for a hack session at SOTM-US. Let 
me know if you are interested so I can find a room for people to meet 
on July 25th.


That's not a bad idea. Sounds like there's some interest in the OSM 
community, and I'd be happy to take a look at it again. I'd say go ahead!


- Lars


___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM

[Clifford Snow]

TopOSM looks like a good candidate for a hack session at SOTM-US. Let me
know if you are interested so I can find a room for people to meet on July
25th.

On Sat, May 21, 2016 at 9:37 AM, Bill Ricker  wrote:

>
>
> On Sat, May 21, 2016 at 10:06 AM, Lars Ahlzen  wrote:
>
> TopOSM was never rendered on-the-fly. It's just a (very large) set of
>> static tiles (currently hosted by Stamen).
>> ​
>>
>
> ​And that's appreciated.​
>
> There's already the OSM cycle map which has a lot of the same features,
>> though with a slightly different focus. I guess one advantage of something
>> like toposm is that it can use higher-resolution data from sources like
>> USGS, and uses US conventions for things like units, symbols and other
>> cartographic details.
>>
>
> ​Yes and yes
> ​
>
>
>> Would it be would be worth picking it up again?
>>
>
> yes, but which goal? ​reordered:
> ​
>
>
>> ​​To update it I'd have to either re-render and upload the entire set, or
>> improve it until it can be rendered in real time. I was working on the
>> latter [1] but never quite finished it.​
>>
>>
> ​If the horsepower to render it real-time is available, it would be nice
> to have, but seems unnecessary.​
> Cyclically updating periodically on a background process would likely be
> good enough.
> For 'real time', cycle-map is good enough.
> The elegance of TopOsm is worth waiting for.​
>
> --
> Bill Ricker
> bill.n1...@gmail.com
> https://www.linkedin.com/in/n1vux
>
> ___
> Talk-us mailing list
> Talk-us@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-us
>
>


-- 
@osm_seattle
osm_seattle.snowandsnow.us
OpenStreetMap: Maps with a human touch
___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM

[Bill Ricker]

On Sat, May 21, 2016 at 10:06 AM, Lars Ahlzen  wrote:

TopOSM was never rendered on-the-fly. It's just a (very large) set of
> static tiles (currently hosted by Stamen).
> ​
>

​And that's appreciated.​

There's already the OSM cycle map which has a lot of the same features,
> though with a slightly different focus. I guess one advantage of something
> like toposm is that it can use higher-resolution data from sources like
> USGS, and uses US conventions for things like units, symbols and other
> cartographic details.
>

​Yes and yes
​


> Would it be would be worth picking it up again?
>

yes, but which goal? ​reordered:
​


> ​​To update it I'd have to either re-render and upload the entire set, or
> improve it until it can be rendered in real time. I was working on the
> latter [1] but never quite finished it.​
>
>
​If the horsepower to render it real-time is available, it would be nice to
have, but seems unnecessary.​
Cyclically updating periodically on a background process would likely be
good enough.
For 'real time', cycle-map is good enough.
The elegance of TopOsm is worth waiting for.​

-- 
Bill Ricker
bill.n1...@gmail.com
https://www.linkedin.com/in/n1vux
___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM

[Marc Gemis]

there is also https://opentopomap.org, is this similar to what you need ?

regards

m

On Sat, May 21, 2016 at 4:06 PM, Lars Ahlzen  wrote:
> On 05/20/2016 10:58 PM, Richard Welty wrote:
>>
>> On 5/20/16 9:24 PM, Clifford Snow wrote:
>>>
>>> I was just on TopOSM [1] which appears to be very outdated. Does
>>> anyone know who maintains this site?
>>>
>>> [1] http://toposm.ahlzen.com/
>>
>> lars ahlzen maintains it. not sure if he's still monitoring this list
>> closely or not.
>
>
> I do. It's pretty out of date indeed.
>
> TopOSM was never rendered on-the-fly. It's just a (very large) set of static
> tiles (currently hosted by Stamen). To update it I'd have to either
> re-render and upload the entire set, or improve it until it can be rendered
> in real time. I was working on the latter [1] but never quite finished it.
>
> There's already the OSM cycle map which has a lot of the same features,
> though with a slightly different focus. I guess one advantage of something
> like toposm is that it can use higher-resolution data from sources like
> USGS, and uses US conventions for things like units, symbols and other
> cartographic details.
>
> Would it be would be worth picking it up again?
>
> - Lars
>
> [1] https://github.com/Ahlzen/TopOSM2
>
> ___
> Talk-us mailing list
> Talk-us@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-us

___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM

[Tod Fitch]

> On May 21, 2016, at 7:06 AM, Lars Ahlzen  wrote:
> 
> On 05/20/2016 10:58 PM, Richard Welty wrote:
>> On 5/20/16 9:24 PM, Clifford Snow wrote:
>>> I was just on TopOSM [1] which appears to be very outdated. Does
>>> anyone know who maintains this site?
>>> 
>>> [1] http://toposm.ahlzen.com/
>> lars ahlzen maintains it. not sure if he's still monitoring this list
>> closely or not.
> 
> I do. It's pretty out of date indeed.
> 
> TopOSM was never rendered on-the-fly. It's just a (very large) set of static 
> tiles (currently hosted by Stamen). To update it I'd have to either re-render 
> and upload the entire set, or improve it until it can be rendered in real 
> time. I was working on the latter [1] but never quite finished it.
> 
> There's already the OSM cycle map which has a lot of the same features, 
> though with a slightly different focus. I guess one advantage of something 
> like toposm is that it can use higher-resolution data from sources like USGS, 
> and uses US conventions for things like units, symbols and other cartographic 
> details.
> 
> Would it be would be worth picking it up again?
> 
> - Lars
> 
> [1] https://github.com/Ahlzen/TopOSM2
> 
> 

I like the style used to generate it and if it were current as far as OSM data 
would use it and recommend it to others.

Thanks!

-Tod




smime.p7s
Description: S/MIME cryptographic signature
___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM

[Lars Ahlzen]

On 05/20/2016 10:58 PM, Richard Welty wrote:

On 5/20/16 9:24 PM, Clifford Snow wrote:

I was just on TopOSM [1] which appears to be very outdated. Does
anyone know who maintains this site?

[1] http://toposm.ahlzen.com/

lars ahlzen maintains it. not sure if he's still monitoring this list
closely or not.


I do. It's pretty out of date indeed.

TopOSM was never rendered on-the-fly. It's just a (very large) set of 
static tiles (currently hosted by Stamen). To update it I'd have to 
either re-render and upload the entire set, or improve it until it can 
be rendered in real time. I was working on the latter [1] but never 
quite finished it.


There's already the OSM cycle map which has a lot of the same features, 
though with a slightly different focus. I guess one advantage of 
something like toposm is that it can use higher-resolution data from 
sources like USGS, and uses US conventions for things like units, 
symbols and other cartographic details.


Would it be would be worth picking it up again?

- Lars

[1] https://github.com/Ahlzen/TopOSM2

___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM

[Clifford Snow]

I left a message for Lars. Hopefully we can find a way to keep TopOSM
uptodate.

Hope to see all at State of the Map US.

On Fri, May 20, 2016 at 7:58 PM, Richard Welty 
wrote:

> On 5/20/16 9:24 PM, Clifford Snow wrote:
> > I was just on TopOSM [1] which appears to be very outdated. Does
> > anyone know who maintains this site?
> >
> > [1] http://toposm.ahlzen.com/
>
> lars ahlzen maintains it. not sure if he's still monitoring this list
> closely or not.
>
> richard
>
> --
> rwe...@averillpark.net
>  Averill Park Networking - GIS & IT Consulting
>  OpenStreetMap - PostgreSQL - Linux
>  Java - Web Applications - Search
>
>
>
> ___
> Talk-us mailing list
> Talk-us@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-us
>
>


-- 
@osm_seattle
osm_seattle.snowandsnow.us
OpenStreetMap: Maps with a human touch
___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM

[Richard Welty]

On 5/20/16 9:24 PM, Clifford Snow wrote:
> I was just on TopOSM [1] which appears to be very outdated. Does
> anyone know who maintains this site?
>
> [1] http://toposm.ahlzen.com/

lars ahlzen maintains it. not sure if he's still monitoring this list
closely or not.

richard

-- 
rwe...@averillpark.net
 Averill Park Networking - GIS & IT Consulting
 OpenStreetMap - PostgreSQL - Linux
 Java - Web Applications - Search




signature.asc
Description: OpenPGP digital signature
___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

[Talk-us] TopOSM

[Clifford Snow]

I was just on TopOSM [1] which appears to be very outdated. Does anyone
know who maintains this site?

[1] http://toposm.ahlzen.com/

-- 
@osm_seattle
osm_seattle.snowandsnow.us
OpenStreetMap: Maps with a human touch
___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM 2

[Lars Ahlzen]

On 02/25/2013 03:06 PM, Rick Marshall wrote:

Serge is right though, the blue text with the white halo for the water
features is difficult to read.  Sometimes it appears a little blurry.
I am not sure a black halo will do the trick either.  What about
changing the color of the text; maybe making it a darker shade of
blue?


I completely agree. This is far from the finished map and it needs a lot 
more work. In addition to hard-to-read labels and layering issues, there 
are many basic features still missing from the map.


Most importantly, the demo uses all pre-rendered tiles. That works for a 
small area like this, but becomes impractical for a nationwide map. The 
biggest challenge may be to get acceptable rendering performance so that 
tiles can be rendered as they are requested. Nevertheless, I'm hopeful 
it can be done without too many compromises.


- Lars


___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM 2

[Rick Marshall]

I agree with the previous replies.  It looks wonderful.  The
cartography is beautiful.

Serge is right though, the blue text with the white halo for the water
features is difficult to read.  Sometimes it appears a little blurry.
I am not sure a black halo will do the trick either.  What about
changing the color of the text; maybe making it a darker shade of
blue?

I love the hillshading.  It makes the contours "pop".

And your project covers my old stomping grounds.  Couldn't like it any better.

Rick Marshall

-- 
Rick Marshall, PhD, GISP
President
Vertical GeoSolutions, Inc (VerticalGeo)
130 Sawgrass Ln
O'Fallon, IL  62269
(618) 670-4259
rick.marsh...@verticalgeo.com
http://www.verticalgeo.com
http://www.culturescapes.net
Vertically Thinking Blog: http://verticalgeo.wordpress.com

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM 2

[dies38061]

Agreed -- looks very nice.  Thank you.  It would be very useful, I think, to 
have a layer which presents the currently available USGS topomaps for two 
reasons:  first, as a reference to illustrate improvements; second, as a 
reference to illustrate gaps.  Also, I should know, but what is the origin of 
the elevation data to provide the hillshading?  Thanks. --Ceyockey

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM 2

[Serge Wroclawski]

Looks very pretty, and I think this would be a great thing to have on
the new US servers!

One small styling thing... It's sometimes a bit hard to read the text.
Could you replace the white halo with a black one, maybe?

- Serge

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

[Talk-us] TopOSM 2

[Lars Ahlzen]

Hi all!

Since Ian Dees recently brought up the OSM-US servers, I thought I'd 
mention that I'm reviving my old TopOSM project which - for those who 
haven't seen it - is a US-wide topo map using OSM data [1]. This time 
it's updated with carto styling, tilestache compositing and support for 
on-the-fly rendering.


Ian already helped me set up a small demo:

http://www.openstreetmap.us/~lahlzen/toposm2.html 



It's a little rough around the edges and still missing many features, 
but I think it could become useful. I was playing with the idea of 
eventually offering this as a US-specific map layer on the OSM-US 
server. Does this sound like a good idea?


Besides the cartography, a fair amount of performance work will be 
needed to make this render fast enough. If anyone is up for a challenge, 
feel free to join in!


The code is on github:

https://github.com/Ahlzen/TopOSM2

- Lars


[1] Mostly, anyway. It's using NED to generate hillshading and contours, 
and NHD for most hydrographic features (though the latter may eventually 
switch back to OSM).



___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM Colorado

[Lars Ahlzen]

Lars Ahlzen wrote:
> Either way, I guess the thing for me to do would be to clean things up 
> just enough to be readable and then put the code and instructions up on 
> the wiki. Then we can all play with it. :)

If you want to try it out, I put the scripts and other files up at 
http://toposm.com/src

There are some very brief instructions at 
http://wiki.openstreetmap.org/wiki/TopOSM_howto

It's still pretty unpolished, but should be enough for the adventurous. :)

Let me know if I forgot anything.

- Lars

-- 
Lars Ahlzen
l...@ahlzen.com

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM Colorado

[Lars Ahlzen]

Jason R. Surratt wrote:
> If you're entertaining the world you may want to look at the Aster GDEM 
> dataset too (http://asterweb.jpl.nasa.gov/gdem.asp). It provides 1arcsec 
> (~30m) post spacing for nearly the entire globe (83N/S) and is freely 
> available. Unfortunately downloading the entire data set from their 
> website would be difficult at best and there are reports of some 
> spurious artifacts. 

That may indeed be the best option for areas outside of the US.

IIRC, there were some questions about licensing, and their servers were 
more or less down at the time, so I haven't tried it. I tried STRM data, 
which has at least comparable resolution/quality, with good results, so 
using Aster should be fairly straightforward.

- Lars

-- 
Lars Ahlzen
l...@ahlzen.com

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM Colorado

[Lars Ahlzen]

Ian Dees wrote:
> The hi-res NHD data set fits on 4.5 DVDs. They sent me a set of NHD data 
> last year around this time for free (they're very nice folks!). I was 
> considering asking nicely for another set, but I was going to try and 
> build a faster computer to handle it, first.

Nice. That would save a lot of time. I hope they can do me the same favor.

> I don't know about the NED data, but I imagine it would be a similar 
> size for the US continental area.

NED is significantly larger, but someone mentioned that one can ship a 
USB hard drive to USGS, they'll fill it with the data and ship it back.

- Lars

-- 
Lars Ahlzen
l...@ahlzen.com

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM Colorado

[Lars Ahlzen]

Apollinaris Schoell wrote:
> 
> This sounds like an interesting experiment worth running on Amazon's
> EC2 cloud. Their x-large machines run at $1/hour, but sure would
> chomp on this data quickly!
> 
> I would be willing to try (and pay for) it if Lars went through a
> did an updated setup doc.
> 
> We could start our own distributed rendering in a similar fashion as 
> ti...@home

Both of those sound like interesting options. Something like E2C might 
be easier to set up (we'd need some more work to split the large data 
sets for a distributed rendering system).

EC2 (or something equivalent) might still be a significant cost, though. 
Looking at the EC2 x-large vm specs, it seems to be about 5-10 times the 
speed of my quad-core rendering server. Import, preprocessing and 
rendering (almost entirely CPU-bound) took just under a week, so it 
might take, let's say, 20 hours on the EC2 wm, or about $20. In that 
case 50 states (assuming approximately equal size) would be $1000. Per 
rendering.

Either way, I guess the thing for me to do would be to clean things up 
just enough to be readable and then put the code and instructions up on 
the wiki. Then we can all play with it. :) I'm right in the middle of a 
move, so it may take a bit longer than otherwise, but I'll do my best.

- Lars

-- 
Lars Ahlzen
l...@ahlzen.com

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM Colorado

[Apollinaris Schoell]

>
>
> This sounds like an interesting experiment worth running on Amazon's EC2
> cloud. Their x-large machines run at $1/hour, but sure would chomp on this
> data quickly!
>
> I would be willing to try (and pay for) it if Lars went through a did an
> updated setup doc.
>
>
We could start our own distributed rendering in a similar fashion as
ti...@home
___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM Colorado

[William Stearns]

Good day, all,

On Fri, 4 Sep 2009, Ian Dees wrote:


On Fri, Sep 4, 2009 at 12:44 PM, Lars Ahlzen  wrote:
  There's also the size of the data sets. The entire hi-res NHD and NED
  would probably be many, many TB. Unpacking, preparing and indexing that
  data requires plenty of additional disk space as well. One would
  probably have to re-work the scripts to work on smaller areas at a time,
  as well as invest in some sizable drives. The tiles themselves for CO
  are about 12 GB and 20 GB for MA, so I'd need a web host that's willing
  to host a TB or so of images. :)


The hi-res NHD data set fits on 4.5 DVDs. They sent me a set of NHD data last 
year around
this time for free (they're very nice folks!). I was considering asking nicely 
for another
set, but I was going to try and build a faster computer to handle it, first.

I don't know about the NED data, but I imagine it would be a similar size for 
the US
continental area.
 
  Finally, Colorado alone took the better part of a week on my quad-core
  rendering server. And this excludes all importing and preprocessing of
  the data. Extrapolate for 48 more states..."


This sounds like an interesting experiment worth running on Amazon's EC2 cloud. 
Their
x-large machines run at $1/hour, but sure would chomp on this data quickly!


	I've had good luck with www.rackspacecloud.com.  I'm using a 4 
core/1GB ram/40GB disk virtual machine running tilesathome and it's easy 
and relatively inexpensive (6 cents/hour).  Their high end vm (16GB 
ram/620GB disk) is also about a dollar/hour:


http://www.rackspacecloud.com/images/cloudservers/serverSizes.png


I would be willing to try (and pay for) it if Lars went through a did an 
updated setup doc.


I'd be willing to as well.


Because the layers are rendered separately, the less-frequently-changing tiles 
(like NED
and for a while the NHD) could be done once for the entire country to save on 
rendering
costs.


Cheers,
- Bill

---
"She worked with a subdued intensity... She once told me that the
only way to know when you have done something truly great is when your
spine tingles."
- on Alice Kober, cryptanalist, in The Code Book, Simon Singh.
--
William Stearns (wstea...@pobox.com, tools and papers: www.stearns.org)
Top-notch computer security training at www.sans.org , www.giac.net
--___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM Colorado

[Ian Dees]

On Fri, Sep 4, 2009 at 12:44 PM, Lars Ahlzen  wrote:

> There's also the size of the data sets. The entire hi-res NHD and NED
> would probably be many, many TB. Unpacking, preparing and indexing that
> data requires plenty of additional disk space as well. One would
> probably have to re-work the scripts to work on smaller areas at a time,
> as well as invest in some sizable drives. The tiles themselves for CO
> are about 12 GB and 20 GB for MA, so I'd need a web host that's willing
> to host a TB or so of images. :)
>

The hi-res NHD data set fits on 4.5 DVDs. They sent me a set of NHD data
last year around this time for free (they're very nice folks!). I was
considering asking nicely for another set, but I was going to try and build
a faster computer to handle it, first.

I don't know about the NED data, but I imagine it would be a similar size
for the US continental area.


> Finally, Colorado alone took the better part of a week on my quad-core
> rendering server. And this excludes all importing and preprocessing of
> the data. Extrapolate for 48 more states..."
>

This sounds like an interesting experiment worth running on Amazon's EC2
cloud. Their x-large machines run at $1/hour, but sure would chomp on this
data quickly!

I would be willing to try (and pay for) it if Lars went through a did an
updated setup doc.

Because the layers are rendered separately, the less-frequently-changing
tiles (like NED and for a while the NHD) could be done once for the entire
country to save on rendering costs.
___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM Colorado

[Lars Ahlzen]

Bill Ricker wrote:
> What, MASSGIS doesn't cover Aspen?

Yup... too bad... :)

>> * Contour lines and hillshading generated from NED [1].
> 
> does that mean you're interpolating lines from a grid?

Yes.

>> * Hydrography (lakes, rivers, wetlands etc) from NHD [2].
> 
> which will eventually be in OSM

Yeah, and it looks like some good progress is being made. But until it's 
complete, I have to use the NHD data directly.

>> * The hillshading, contour lines and map features are on separate
>> layers. Use the layer switcher (top right "+") to toggle.
> 
> AWESOME. that's beautiful.

The main downside is that it increases the number of map tiles that have 
to be downloaded and makes scrolling choppy on slower hardware/browsers.

I think it's worth it, though.

>> Since the data is nationwide, the same technique could (theoretically)
>> be used to generate a complete TopOSM map of the US, or even the world
>> by using e.g. SRTM for elevation and OSM for hydrography.
> 
> requiring only a render farm plus some config ?

Yeah... well... for certain values of "some". It would actually require 
a fair amount of work. Here's my reply to another OSMapper asking me 
essentially the same question:

"That said, there are some obstacles to overcome before it can happen. 
For example:

A lot of manual labor is spent on downloading the data from USGS. Their 
java point-and-click interface (with lots of limitations on batch sizes 
etc) is not helpful for what I do, and nobody seem to mirror the data 
sets in a better format.

There's also the size of the data sets. The entire hi-res NHD and NED 
would probably be many, many TB. Unpacking, preparing and indexing that 
data requires plenty of additional disk space as well. One would 
probably have to re-work the scripts to work on smaller areas at a time, 
as well as invest in some sizable drives. The tiles themselves for CO 
are about 12 GB and 20 GB for MA, so I'd need a web host that's willing 
to host a TB or so of images. :)

Finally, Colorado alone took the better part of a week on my quad-core 
rendering server. And this excludes all importing and preprocessing of 
the data. Extrapolate for 48 more states..."

The above should not stop me from trying, though...


> Use any perl in ingesting or anything?

Wish I could say yes, but I'd be lying... it's all Python and Bash. :(

- Lars

-- 
Lars Ahlzen
l...@ahlzen.com

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM Colorado

[Chris Wold, PhD]

M, Ian Dees wrote:
> > No tag should ever imply any other tag. It's always better to be more
> > verbose than not.
> 
> No it's not.  Are you seriously putting oneway=no (just to name one
> example) on every street you tag?
> 
> -Alex Mauer "hawke"
> 
> -- next part --
> A non-text attachment was scrubbed...
> Name: signature.asc
> Type: application/pgp-signature
> Size: 261 bytes
> Desc: OpenPGP digital signature
> Url : http://lists.openstreetmap.org/pipermail/talk-
> us/attachments/20090903/55f71eb0/attachment-0001.pgp
> 
> --
> 
> Message: 5
> Date: Thu, 3 Sep 2009 13:44:16 -0500
> From: Ian Dees 
> Subject: Re: [Talk-us] Unpaved streets
> To: Alex Mauer 
> Cc: talk-us@openstreetmap.org
> Message-ID:
>   <3baad6b80909031144p1808f421t484775d7d8200...@mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
> 
> On Thu, Sep 3, 2009 at 1:30 PM, Alex Mauer  wrote:
> 
> > On 09/03/2009 09:17 AM, Ian Dees wrote:
> > > No tag should ever imply any other tag. It's always better to be more
> > > verbose than not.
> >
> > No it's not.  Are you seriously putting oneway=no (just to name one
> > example) on every street you tag?
> 
> 
> Dangit, someone (probably you, Alex :) ) made this point before. You're
> right ... but I don't think of oneway=no as an implied tag. I would call
> it
> the default for all road ways... which I guess is an implied tag.
> 
> Ok, how about this: implied tags should be the exception, not the rule.
> -- next part --
> An HTML attachment was scrubbed...
> URL: http://lists.openstreetmap.org/pipermail/talk-
> us/attachments/20090903/ef0128b3/attachment-0001.htm
> 
> --
> 
> Message: 6
> Date: Thu, 03 Sep 2009 20:15:08 -0400
> From: Lars Ahlzen 
> Subject: [Talk-us] TopOSM Colorado
> To: talk-us@openstreetmap.org
> Message-ID: <4aa05c0c.50...@ahlzen.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> 
> Hi All!
> 
> It's been a while since the last TopOSM update, but I haven't been
> resting. After Massachusetts, I decided to pick a state with somewhat
> more interesting topography: Colorado.
> 
> http://toposm.com/co/
> 
> There are still a few rough edges and things that are missing (like the
> map legend), so consider it a "preview".
> 
> TopOSM-CO has a few important differences from TopOSM-MA:
> 
> * Color-by-elevation in base layer.
> 
> * Contour lines and hillshading generated from NED [1].
> 
> * Hydrography (lakes, rivers, wetlands etc) from NHD [2].
> 
> * The hillshading, contour lines and map features are on separate
> layers. Use the layer switcher (top right "+") to toggle.
> 
> Since the data is nationwide, the same technique could (theoretically)
> be used to generate a complete TopOSM map of the US, or even the world
> by using e.g. SRTM for elevation and OSM for hydrography.
> 
> I'll update the TopOSM wiki page [3] with the changes in this version as
> soon as I can.
> 
> - Lars
> 
> 
> 
> [1] National Elevation Dataset. Available for the entire US.
> http://ned.usgs.gov/
> 
> [2] National Hydrography Dataset. Available for the entire US. There is
> some effort to import this into OSM, but we're not there yet.
> http://nhd.usgs.gov/
> 
> [3] http://wiki.openstreetmap.org/wiki/TopOSM
> 
> --
> Lars Ahlzen
> l...@ahlzen.com
> 
> 
> 
> 
> --
> 
> Message: 7
> Date: Thu, 03 Sep 2009 21:46:17 -0400
> From: Lars Ahlzen 
> Subject: Re: [Talk-us] TopOSM Colorado
> To: talk-us@openstreetmap.org
> Message-ID: <4aa07169.3040...@ahlzen.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> 
> Richard Weait wrote:
> > What I saw was beautiful before we squashed your server! ;-)
> 
> Hmm... I too have noticed that the site occasionally stops responding
> after scrolling the map around a bit. It tends to come back in 5 minutes
> or so.
> 
> TopOSM is currently hosted by a hosting company on some serious
> hardware, so I don't think it's getting overwhelmed. I'm thinking that
> perhaps the very large number of requests in a short time looks like an
> attack and they temporarily block the IP.
> 
> Is anyone else experiencing this?
> 
> - Lars
> 
> --
> Lars Ahlzen
> l...@ahlzen.com
> 
> 
> 
> --
> 
> Message: 8
> Date: Thu, 3 Sep 2009 21:59:15 -0400
> From: Richard Weait 
> Subject: Re: 

Re: [Talk-us] TopOSM Colorado

[Jason R. Surratt]

Thanks. We love it here.

I've been picking away at Manitou over the past month or so -- still a long
way to go, but we're getting there.

-jason

On Thu, Sep 3, 2009 at 11:02 PM, Richard Weait  wrote:

> On Thu, Sep 3, 2009 at 10:54 PM, Jason R.
> Surratt wrote:
> >
> > Absolutely phenomenal. I live in the Colorado Springs area and it makes
> some
> > of the trails I've been mapping really pop. Great job!
>
> You live in a beautiful place, Jason.  If you get a chance, run up to
> Manitou Springs and fix some of the TIGER geometry there?  Such a nice
> place deserves better data. I was able to fix a few things last time I
> was there, but trips are too few and too far between.
>
> Best regards,
> Richard
>
> ___
> Talk-us mailing list
> Talk-us@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/talk-us
>
___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM Colorado

[Richard Weait]

On Thu, Sep 3, 2009 at 10:54 PM, Jason R.
Surratt wrote:
>
> Absolutely phenomenal. I live in the Colorado Springs area and it makes some
> of the trails I've been mapping really pop. Great job!

You live in a beautiful place, Jason.  If you get a chance, run up to
Manitou Springs and fix some of the TIGER geometry there?  Such a nice
place deserves better data. I was able to fix a few things last time I
was there, but trips are too few and too far between.

Best regards,
Richard

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM Colorado

[Jason R. Surratt]

> It's been a while since the last TopOSM update, but I haven't been
> resting. After Massachusetts, I decided to pick a state with somewhat
> more interesting topography: Colorado.
>
> http://toposm.com/co/
>
>
Absolutely phenomenal. I live in the Colorado Springs area and it makes some
of the trails I've been mapping really pop. Great job!

If you're entertaining the world you may want to look at the Aster GDEM
dataset too (http://asterweb.jpl.nasa.gov/gdem.asp). It provides 1arcsec
(~30m) post spacing for nearly the entire globe (83N/S) and is freely
available. Unfortunately downloading the entire data set from their website
would be difficult at best and there are reports of some spurious
artifacts.

-jason
___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM Colorado

[Bill Ricker]

> TopOSM-CO has a few important differences from TopOSM-MA:

What, MASSGIS doesn't cover Aspen?

> * Color-by-elevation in base layer.

interesting

> * Contour lines and hillshading generated from NED [1].

does that mean you're interpolating lines from a grid?

> * Hydrography (lakes, rivers, wetlands etc) from NHD [2].

which will eventually be in OSM

> * The hillshading, contour lines and map features are on separate
> layers. Use the layer switcher (top right "+") to toggle.

AWESOME. that's beautiful.

> Since the data is nationwide, the same technique could (theoretically)
> be used to generate a complete TopOSM map of the US, or even the world
> by using e.g. SRTM for elevation and OSM for hydrography.

requiring only a render farm plus some config ?

Use any perl in ingesting or anything?


-- 
Bill
n1...@arrl.net bill.n1...@gmail.com

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM Colorado

[Richard Weait]

On Thu, Sep 3, 2009 at 9:46 PM, Lars Ahlzen wrote:
> Richard Weait wrote:
>> What I saw was beautiful before we squashed your server! ;-)
>
> Hmm... I too have noticed that the site occasionally stops responding
> after scrolling the map around a bit. It tends to come back in 5 minutes
> or so.
>
> TopOSM is currently hosted by a hosting company on some serious
> hardware, so I don't think it's getting overwhelmed. I'm thinking that
> perhaps the very large number of requests in a short time looks like an
> attack and they temporarily block the IP.
>
> Is anyone else experiencing this?

When I looked later it was still gorgeous.  Right to the top of the
best of OSM list with you!

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM Colorado

[Lars Ahlzen]

Richard Weait wrote:
> What I saw was beautiful before we squashed your server! ;-)

Hmm... I too have noticed that the site occasionally stops responding
after scrolling the map around a bit. It tends to come back in 5 minutes
or so.

TopOSM is currently hosted by a hosting company on some serious
hardware, so I don't think it's getting overwhelmed. I'm thinking that
perhaps the very large number of requests in a short time looks like an
attack and they temporarily block the IP.

Is anyone else experiencing this?

- Lars

-- 
Lars Ahlzen
l...@ahlzen.com

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

[Talk-us] TopOSM Colorado

[Lars Ahlzen]

Hi All!

It's been a while since the last TopOSM update, but I haven't been
resting. After Massachusetts, I decided to pick a state with somewhat 
more interesting topography: Colorado.

http://toposm.com/co/

There are still a few rough edges and things that are missing (like the
map legend), so consider it a "preview".

TopOSM-CO has a few important differences from TopOSM-MA:

* Color-by-elevation in base layer.

* Contour lines and hillshading generated from NED [1].

* Hydrography (lakes, rivers, wetlands etc) from NHD [2].

* The hillshading, contour lines and map features are on separate
layers. Use the layer switcher (top right "+") to toggle.

Since the data is nationwide, the same technique could (theoretically)
be used to generate a complete TopOSM map of the US, or even the world 
by using e.g. SRTM for elevation and OSM for hydrography.

I'll update the TopOSM wiki page [3] with the changes in this version as
soon as I can.

- Lars



[1] National Elevation Dataset. Available for the entire US.
http://ned.usgs.gov/

[2] National Hydrography Dataset. Available for the entire US. There is
some effort to import this into OSM, but we're not there yet.
http://nhd.usgs.gov/

[3] http://wiki.openstreetmap.org/wiki/TopOSM

-- 
Lars Ahlzen
l...@ahlzen.com


___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM update

[John Callahan]

Thank you Lars, for both the excellent maps and your documentation of 
how you created them.  Very nice.  I hope to be doing the same for my 
area (Delaware, U.S.) very soon.  We have a couple of topographic map 
experts around (well, they might not call themselves experts!) and I 
plan to have them look at it.

- John

**
John Callahan
Geospatial Application Developer
Delaware Geological Survey, University of Delaware
227 Academy St, Newark DE 19716-7501
Tel: (302) 831-3584  
Email: john.calla...@udel.edu
http://www.dgs.udel.edu
**




Lars Ahlzen wrote:
> Hi!
>
> It took a while (I've been busy), but I put an updated version of the
> TopOSM map online:
>
> http://toposm.com/ma/
>
> Most of the improvements are minor, but many of them came from you on
> this list, so thanks everyone for your suggestions! There's a brief
> changelog at http://toposm.com/changelog.html
>
> It still has plenty of issues, and I'm (slowly) working on them...
>
> - Lars
>
>   

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM update

[Lars Ahlzen]

Dale Puch wrote:
> I'm not sure which is better, but consider rendering borders on top of
> water.  County and state at least.  There were a few places I think it
> would be more informative that way.

I agree. For some reason, I missed that in the last rendering.

Btw, state and county borders in Massachusetts are a bit of a mess, so
some cleanup of the actual OSM data might be necessary for these to be
useful.

- Lars

-- 
Lars Ahlzen
l...@ahlzen.com

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM update

[Theodore Book]

Very impressive!  It is one of the nicest looking maps on the web in my 
opinion.

Lars Ahlzen wrote:
> Hi!
> 
> It took a while (I've been busy), but I put an updated version of the
> TopOSM map online:
> 
> http://toposm.com/ma/
> 
> Most of the improvements are minor, but many of them came from you on
> this list, so thanks everyone for your suggestions! There's a brief
> changelog at http://toposm.com/changelog.html
> 
> It still has plenty of issues, and I'm (slowly) working on them...
> 
> - Lars
> 


___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

Re: [Talk-us] TopOSM update

[Dale Puch]

I'm not sure which is better, but consider rendering borders on top of
water.  County and state at least.  There were a few places I think it
would be more informative that way.

Dale

On Sat, May 23, 2009 at 7:18 PM, Lars Ahlzen  wrote:
> Hi!
>
> It took a while (I've been busy), but I put an updated version of the
> TopOSM map online:
>
> http://toposm.com/ma/
>
> Most of the improvements are minor, but many of them came from you on
> this list, so thanks everyone for your suggestions! There's a brief
> changelog at http://toposm.com/changelog.html
>
> It still has plenty of issues, and I'm (slowly) working on them...
>
> - Lars
>
> --
> Lars Ahlzen
> l...@ahlzen.com
>
> ___
> Talk-us mailing list
> Talk-us@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/talk-us
>



-- 
Dale Puch

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

[Talk-us] TopOSM update

[Lars Ahlzen]

Hi!

It took a while (I've been busy), but I put an updated version of the
TopOSM map online:

http://toposm.com/ma/

Most of the improvements are minor, but many of them came from you on
this list, so thanks everyone for your suggestions! There's a brief
changelog at http://toposm.com/changelog.html

It still has plenty of issues, and I'm (slowly) working on them...

- Lars

-- 
Lars Ahlzen
l...@ahlzen.com

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us

[Talk-us] TopOSM wiki page

[Lars Ahlzen]

Since several people asked for the details of how it was created, I
added a wiki page for TopOSM (the Mass topo map). It's a draft, and
probably full of errors, but at least it's a start.

I included some illustrations of the techniques used, as well as (I
think) all of the scripts and other details required to actually
generate it, in case someone feels adventurous... :)

http://wiki.openstreetmap.org/wiki/TopOSM

- Lars

-- 
Lars Ahlzen
l...@ahlzen.com

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us