Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom Scenery branch, gdalchop,
Flightgear-commitlogs wrote: The branch, gdalchop has been created at 40695c40585943b13e3c1005648e1da4dc490542 (commit) - Log - commit 40695c40585943b13e3c1005648e1da4dc490542 Author: Ralf Gerlich Date: Tue Dec 29 19:04:46 2009 +0100 Sorry for the old changelogs. Apparently the history in the 'official' terragear-cs repo on Gitorious was bent forcibly and therefore some of the references didn't apply properly to the mirror on MapServer - at least GIT automatically detects these 'excercises' :-) I looks like branch removals got lost as well, I'll try to fix these later. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
On Friday, January 20, 2012 07:55:24 Martin Spott wrote: Martin Spott wrote: Topologically cleaned CLC2000v15 is now available on the FlightGear MapServer web map and if you download Shapefiles, you'll get them from the revised dataset. Please let me know if you encounter technical mistakes in this data ! CLC2006v15 likewise, Martin. Hi Martin, That's great news. I'm curious, what kind of elevation data are you using for the new terrain? SRTMv3 by any chance? Cheers, Adrian -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
On Fri, Jan 20, 2012 at 12:09 AM, Martin Spott martin.sp...@mgras.netwrote: A agree with all of the above. Creating all four edges for a stand- alone tile sounds reasonable and creating just N/E when S/W are available isn't that bad either. Anyhow, as far as I understand, fgfs-construct was made for a workflow where _at_minimum_ two new edges are being created with a new tile. It's missing the feature to smoothly align a tile with a surrounding box of four edges which are given from a previous terrain build. Trying to insert a single tile, or a chunk of just a few tiles connected tiles results in strange artifacts in almost every second case. That is curious -- it should do fine in that situation. Originally the tile scheduler would build every other tile in every other row, and then come back and file in so eventually it should have hit this situation many times. My thinking was that in a typical situation with let's say 25 build nodes, if the scheduler assigned adjacent tiles to the build nodes, there could be a race condition when tiles decide if they own an edge or not. But by interleaving the schedule, there should never be a situation where two build nodes are ever working on adjacent tiles. The behavior you are describing definitely sounds like (a) a bug, or (b) constructing an isololated/surrounded tile with new land cover data such that it's impossible to get a correct match with the adjacent neighbor (built with different polygon data.) Situation (b) could easily happen if you are changing data or changing build options compared to the previous build. Curt. -- Curtis Olson: http://www.atiak.com - http://aem.umn.edu/~uav/ http://www.flightgear.org - http://gallinazo.flightgear.org -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
Adrian Musceac wrote: That's great news. I'm curious, what kind of elevation data are you using for the new terrain? In general I think I'll try to use the best I can get under a GPL- compatible license ;-) SRTMv3 by any chance? Well, one day I'll have to make a decision but beforehand the most pressing item on my TODO list is to get the land cover right. Which SRTMv3 are you having in mind ? The only SRTMv3 I know is the first version of CGIAR's void-filled SRTM v2.1 dataset - which, btw., has a no commercial use-clause in its terms of use. The same applies to CGIAR v4 and v4.1. Anyhow, nowadays there's no shortage of alternative supplements to fill the voids in SRTM v2.1, therefore I feel quite relaxed wrt. the choice of DEM data. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
Martin Spott wrote: CLC2006v15 likewise, BTW, here's my list of most urgent open items related to the web map which I think are worth being addressed - maybe someone would like to volunteer: - Make the four input fields and the download button disappear as long as the Download-Box layer is inactive. - Enable blending between two different layers, most particularly between VMap0 line features and OSM in order to identify voids in OSM where VMap0 has at least basic road coverage (to support the preparations for switching over to OSM as the primary source of road data). - Clean up the mess of having one pair of download scripts for every of the download features. Merge these scripts into one or two pairs while retaining the security features. The code is here, feel free to submit merge requests. http://www.gitorious.org/fg/sceneryweb/trees/master/mapserver/ Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
On Fri, 2012-01-20 at 12:29 +, Martin Spott wrote: Adrian Musceac wrote: That's great news. I'm curious, what kind of elevation data are you using for the new terrain? In general I think I'll try to use the best I can get under a GPL- compatible license ;-) SRTMv3 by any chance? Well, one day I'll have to make a decision but beforehand the most pressing item on my TODO list is to get the land cover right. Which SRTMv3 are you having in mind ? The only SRTMv3 I know is the first version of CGIAR's void-filled SRTM v2.1 dataset - which, btw., has a no commercial use-clause in its terms of use. The same applies to CGIAR v4 and v4.1. Anyhow, nowadays there's no shortage of alternative supplements to fill the voids in SRTM v2.1, therefore I feel quite relaxed wrt. the choice of DEM data. Cheers, Martin. Martin, I dont know if this is possible but due to the low resolution of hight data available for my place of interest (Australia) I have been thinking of using the contour layer of the available shape files from ga.gov.au Several years ago I experimented with this using a script to build a file for Fred's FGSD program and the results were quite promising although I had issues as the old STRM data was already there. The scenery that I did produce look very pleasing as the local cliffs where all visible due to the 10m contouring in the shape files. Maybe this might be an avenue to look at again. Jason -- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
Martin Spott wrote: Martin Spott wrote: Supplying the flags --xdist=19 --ydist=18 (around a center at --lon=9.875 --lat=49.375, but there's no difference with --lon=10 --lat=49) seem to work nicely (test is still running, but looks good so far), [...] Test is still running and has produced 1.8 GByte of terrain so far. Finally the test created 3.3 GByte of terrain with CORINE land cover and VMap0 roads in 49 hours using one single process, before it ran into a segfault. I think that's still pretty good and therefore I'd declare this test a success, despite the segfault. Cleaning VMap0 proved not to be that successful after a close inspection, I'm currently re-doing that one with different parameters (much smaller snapping threshold). As soon as VMap0 looks reasonable (maybe in a week or three, who knows), I'll start merging CORINE and all the hand-carved landcover - at least those datasets I can get my hands on Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
Curtis Olson wrote: On Tue, Jan 17, 2012 at 11:49 AM, Martin Spott martin.sp...@mgras.netwrote: 1.) As far as I can tell there's still no reliable solution for creating proper, smooth seams between a newly built terrain tile and the adjacent, old tiles. The fgfs-construct honours two flags --no-write-shared-edges and --use-own-shared-edges which were meant to serve as a solution when the Shared directory of the previous full terrain build is available, but the underlying mechanics don't work in every case. Sometimes the success seems to depend on how flat the terrain is, sometimes the result looks just random. The original code saved a directory tree of all the shared edges between tiles. The tile that is built first is the one that gets to define the points along the shared edge and all subsequent tiles must use those. Yup, using this Shared directory is still state-of-the-art. Anyhow, this machinery works only in the limited case where just one or two edges have to get seamed together - the western and the southern edge of the new tile. It doesn't work for the eastern and the northern edge because the traditional fgfs-construct was designed to create these edges from scratch with the new tile. The original intent was for fgfs-construct to be called once per tile [...] Sure, for a World Scenery build I'd get back to distributing the terrain compilation over different nodes (the old hypersphere was able to construct worldwide terrain in approx. 30 hours on four CPU sockets). Anyhow, from a technical point of view, having a tool which is capable of creating the world terrain in a single process without memory leaks and without segfaults would be preferred ;-) We all like bug free code, but logically correct code can get thrown for a loop when you give it arbitrary limited precision numeric/floating point data. Hey, that's why I'm wasting my (and other people's) time for providing topologically and numerically clean data. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
Hi Jason, Jason Cox wrote: On Tue, 2012-01-17 at 17:49 +, Martin Spott wrote: Supplying the flags --xdist=19 --ydist=18 (around a center at --lon=9.875 --lat=49.375, but there's no difference with --lon=10 --lat=49) seem to work nicely (test is still running, but looks good so far), but just swapping the numbers for xdist and ydist or selecting any bigger number for either of them results in fgfs-construct pretending to having finished successfully, but writing not more than approx. 30 MByte of terrain tiles to the disk. this sounds like the memory leak that I reported a few months ago that no one else could see. If you watch the memory use during the build I suspect that as each tile is processed it memory allocation will remain thus by the time you get to the end there is no more memory and the process fails. I'm not certain if we're looking at the same issue. The case I'm describing above typically runs for less than 6 minutes ! and barely consumes any memory. Even the large single job which ended this evening, creating 3 GByte of terrain in more than two days !! didn't run out of memory, no oom killer was stepping in. It just ran into a segfault, which, as you certainly know, is a pretty common case. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
On Thu, Jan 19, 2012 at 4:01 PM, Martin Spott martin.sp...@mgras.netwrote: Yup, using this Shared directory is still state-of-the-art. Anyhow, this machinery works only in the limited case where just one or two edges have to get seamed together - the western and the southern edge of the new tile. It doesn't work for the eastern and the northern edge because the traditional fgfs-construct was designed to create these edges from scratch with the new tile. My recollection is that the original fgfs-construct created any shared edges that weren't already created. So if free standing tile was created before any of its neighbors, all the shared edges (and corners) should be created: N, S, E, and W. If that's not the case, then perhaps I'm misunderstanding your comments, or something got changed in the code along the way. Note that if you march along creating tiles in regular strips, then after you've completely one row, and after you've completed the first tile of the next row, then each new tile would create just two new edges (so perhaps that is what you are seeing?) Regards, Curt. -- Curtis Olson: http://www.atiak.com - http://aem.umn.edu/~uav/ http://www.flightgear.org - http://gallinazo.flightgear.org -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
Martin Spott wrote: Topologically cleaned CLC2000v15 is now available on the FlightGear MapServer web map and if you download Shapefiles, you'll get them from the revised dataset. Please let me know if you encounter technical mistakes in this data ! CLC2006v15 likewise, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
Curtis Olson wrote: My recollection is that the original fgfs-construct created any shared edges that weren't already created. So if free standing tile was created before any of its neighbors, all the shared edges (and corners) should be created: N, S, E, and W. If that's not the case, then perhaps I'm misunderstanding your comments, or something got changed in the code along the way. Note that if you march along creating tiles in regular strips, then after you've completely one row, and after you've completed the first tile of the next row, then each new tile would create just two new edges (so perhaps that is what you are seeing?) A agree with all of the above. Creating all four edges for a stand- alone tile sounds reasonable and creating just N/E when S/W are available isn't that bad either. Anyhow, as far as I understand, fgfs-construct was made for a workflow where _at_minimum_ two new edges are being created with a new tile. It's missing the feature to smoothly align a tile with a surrounding box of four edges which are given from a previous terrain build. Trying to insert a single tile, or a chunk of just a few tiles connected tiles results in strange artifacts in almost every second case. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
Martin Spott wrote: Supplying the flags --xdist=19 --ydist=18 (around a center at --lon=9.875 --lat=49.375, but there's no difference with --lon=10 --lat=49) seem to work nicely (test is still running, but looks good so far), [...] Test is still running and has produced 1.8 GByte of terrain so far. I felt slightly frustrated when I came across a huge hole in the terrain while test-flying the result of the above build - until I realized that Andorra is not covered in CORINE :-) Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
James Turner wrote: Please let us know what help you need to make it happen, [...] Just a quick message for a better understanding of where trouble is still hiding: 1.) As far as I can tell there's still no reliable solution for creating proper, smooth seams between a newly built terrain tile and the adjacent, old tiles. The fgfs-construct honours two flags --no-write-shared-edges and --use-own-shared-edges which were meant to serve as a solution when the Shared directory of the previous full terrain build is available, but the underlying mechanics don't work in every case. Sometimes the success seems to depend on how flat the terrain is, sometimes the result looks just random. 2.) I think I've successfully built CORINE-based terrain on a single fgfs-construct process up to the size of --xdist=10 --ydist=10, (which covers an area of 20 degree longitude and latitude). I think, because I didn't fly the terrain in FG, but on the disk it's been a chunk of plausible size. In order to let fgfs-construct process such a large section on a single run, I've removed the entire cpu-time and mem-alloc limit section from src/BuildTiles/Main/main.cxx Surprisingly there seems to be another limit with really large areas. Supplying the flags --xdist=19 --ydist=18 (around a center at --lon=9.875 --lat=49.375, but there's no difference with --lon=10 --lat=49) seem to work nicely (test is still running, but looks good so far), but just swapping the numbers for xdist and ydist or selecting any bigger number for either of them results in fgfs-construct pretending to having finished successfully, but writing not more than approx. 30 MByte of terrain tiles to the disk. Until now I haven't done any search for the cause, just writing down what I found out this afternoon. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
On Tue, Jan 17, 2012 at 11:49 AM, Martin Spott martin.sp...@mgras.netwrote: 1.) As far as I can tell there's still no reliable solution for creating proper, smooth seams between a newly built terrain tile and the adjacent, old tiles. The fgfs-construct honours two flags --no-write-shared-edges and --use-own-shared-edges which were meant to serve as a solution when the Shared directory of the previous full terrain build is available, but the underlying mechanics don't work in every case. Sometimes the success seems to depend on how flat the terrain is, sometimes the result looks just random. The original code saved a directory tree of all the shared edges between tiles. The tile that is built first is the one that gets to define the points along the shared edge and all subsequent tiles must use those. The world tiling scheme used tiles that were a preset width referencing longitude (i.e 1/8th of a degree). But as you proceed towards the poles, tiles get skinnier and skinnier. So at a couple predefined latitudes we expanded the tile width (in terms of longitude, i.e. 1/4 degree, then 1/2 degree.) However we never accounted well for the seam between changing tile widths. Essentially 1 tile would match up with two adjacent tiles and there would be cracks. But other than that unfortunate oversight, any cracks/gaps between tiles would be a bug. Or it would mean that the source data changed for one tile, but not for the adjacent tile. The right thing to do would be to (a) remove the affected shared edges and (b) rebuild all the affected tiles. 2.) I think I've successfully built CORINE-based terrain on a single fgfs-construct process up to the size of --xdist=10 --ydist=10, (which covers an area of 20 degree longitude and latitude). I think, because I didn't fly the terrain in FG, but on the disk it's been a chunk of plausible size. In order to let fgfs-construct process such a large section on a single run, I've removed the entire cpu-time and mem-alloc limit section from src/BuildTiles/Main/main.cxx Surprisingly there seems to be another limit with really large areas. Supplying the flags --xdist=19 --ydist=18 (around a center at --lon=9.875 --lat=49.375, but there's no difference with --lon=10 --lat=49) seem to work nicely (test is still running, but looks good so far), but just swapping the numbers for xdist and ydist or selecting any bigger number for either of them results in fgfs-construct pretending to having finished successfully, but writing not more than approx. 30 MByte of terrain tiles to the disk. It's possible that I included a sanity check for size -- the mem/cpu limit is there because the construction process could hang in an infinite loop for some tile if the source data did not play nice, rather than have a build node hang up infinitely, I wanted to log a failure so I could (a) go back and investigate the specifics of the failure and (b) the build node could leave the problem tile and continue to build other tiles and not be taken out of the game. The original intent was for fgfs-construct to be called once per tile with a distributed network of build nodes and a master server passing out work to any node that became free. Having fgfs-construct build many tiles across a range was more of an after-thought added as a convenience for testing -- it wasn't ever intended to be the primary run mode of fgfs-construct. We all like bug free code, but logically correct code can get thrown for a loop when you give it arbitrary limited precision numeric/floating point data. Curt. -- Curtis Olson: http://www.atiak.com - http://aem.umn.edu/~uav/ http://www.flightgear.org - http://gallinazo.flightgear.org -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
On Tue, 2012-01-17 at 17:49 +, Martin Spott wrote: Surprisingly there seems to be another limit with really large areas. Supplying the flags --xdist=19 --ydist=18 (around a center at --lon=9.875 --lat=49.375, but there's no difference with --lon=10 --lat=49) seem to work nicely (test is still running, but looks good so far), but just swapping the numbers for xdist and ydist or selecting any bigger number for either of them results in fgfs-construct pretending to having finished successfully, but writing not more than approx. 30 MByte of terrain tiles to the disk. Martin, this sounds like the memory leak that I reported a few months ago that no one else could see. If you watch the memory use during the build I suspect that as each tile is processed it memory allocation will remain thus by the time you get to the end there is no more memory and the process fails. Jason Cox -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
James Turner wrote: Of course, this will be fantastic. Please let us know what help you need to make it happen, and thanks for all your long, hidden efforts on this. Topologically cleaned CLC2000v15 is now available on the FlightGear MapServer web map and if you download Shapefiles, you'll get them from the revised dataset. Please let me know if you encounter technical mistakes in this data ! Next I'll build TerraGear using the latest modifications in the main repository and check wether I'm able to create a crash-free terrain for Europe. I'll keep you posted about the result. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- RSA(R) Conference 2012 Mar 27 - Feb 2 Save $400 by Jan. 27 Register now! http://p.sf.net/sfu/rsa-sfdev2dev2 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
Gijs de Rooy wrote: If not, would you mind if I copyedited some of your mail? Please feel free to do everything what you think makes sense. I'm having a (programming) deadline to meet until end of the week, therefore I'm probably not going to do anything which doesn't fit into chunks of 5 minutes (which is my typical FG schedule anyway ;-) Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- RSA(R) Conference 2012 Mar 27 - Feb 2 Save $400 by Jan. 27 Register now! http://p.sf.net/sfu/rsa-sfdev2dev2 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
On 13 Jan 2012, at 17:46, Martin Spott wrote: Finally 3.) As soon as the various tools are working in a predictable manner, I'm planning to revive the central idea, the driving motivation behind all the effort, which is to build an infrastructure for re-compiling the respective terrain tiles on a regular, maybe daily schedule after the land cover has been updated in a certain region. Having a stable and reliable procedure to clean the vector data is one of the vital requirements for such an adventure, therefore I'm quite excited today Of course, this will be fantastic. Please let us know what help you need to make it happen, and thanks for all your long, hidden efforts on this. James -- RSA(R) Conference 2012 Mar 27 - Feb 2 Save $400 by Jan. 27 Register now! http://p.sf.net/sfu/rsa-sfdev2dev2 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
Martin wrote: I'd like to point out that I'm now having what I think should be topologically clean CLC2000v15 and VMap0 edition 5.0 datasets. I'll load these into the Landcover-DB as replacements for the current VMap0 and CLC layers. That's excellent news indeed! Would you write a short update in the Newsletter? If not, would you mind if I copyedited some of your mail? This is something that deserves some more attention and respect ;-) Cheers, Gijs -- RSA(R) Conference 2012 Mar 27 - Feb 2 Save $400 by Jan. 27 Register now! http://p.sf.net/sfu/rsa-sfdev2dev2___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
I'd like to point out that I'm now having what I think should be topologically clean CLC2000v15 and VMap0 edition 5.0 datasets. I'll load these into the Landcover-DB as replacements for the current VMap0 and CLC layers. Thanks to the great folks at GRASS GIS, most notably Markus M., the procedure of cleaning large vector datasets like CORINE is now a matter of just a few days and runs on a machine with far less than 32 GByte of main menory !! One year ago, cleaning CORINE took about one and a half months to complete just one single ! of multiple iterations and you never knew if somebody would reboot the server while the job was still running (which actually happened several times). Having topologically clean datasets is a very important step for at least two reasons: 1.) I expect TerraGear's overall stability to increase with topologically clean vector data. 2.) Merging custom land cover data (like John Holden's great work) into the Custom Scenery layer (at MapServer) had been failing in some cases due to topological inconsistencies. I expect these failures to disappear with topologically clean vector data. I'll be providing a clean CLC2006 as well, but because that one's still incomplete (and work in progress, actually) I think CLC2000 is more important. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- RSA(R) Conference 2012 Mar 27 - Feb 2 Save $400 by Jan. 27 Register now! http://p.sf.net/sfu/rsa-sfdev2dev2 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
Martin Spott wrote: 2.) Merging custom land cover data (like John Holden's great work) into the Custom Scenery layer (at MapServer) had been failing in some cases due to topological inconsistencies. I expect these failures to disappear with topologically clean vector data. Finally 3.) As soon as the various tools are working in a predictable manner, I'm planning to revive the central idea, the driving motivation behind all the effort, which is to build an infrastructure for re-compiling the respective terrain tiles on a regular, maybe daily schedule after the land cover has been updated in a certain region. Having a stable and reliable procedure to clean the vector data is one of the vital requirements for such an adventure, therefore I'm quite excited today Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- RSA(R) Conference 2012 Mar 27 - Feb 2 Save $400 by Jan. 27 Register now! http://p.sf.net/sfu/rsa-sfdev2dev2 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
Martin Spott wrote: I've been using osm2pgsql for years for importing The Planet Dump into PostGIS and so far I'm quite satisfied. After the bare import I'm using a custom script to separate the various road types into distinct tables/layers, something like: landcover= SELECT osm_id, access, admin_level, z_order, way_area, wkb_geometry landcover- INTO osm_motorway landcover- FROM planet_osm_line landcover- WHERE highway LIKE 'motorway%' landcover- AND ST_GeometryType(wkb_geometry) LIKE 'ST_LineString'; That way I don't have to query the entire planet_osm_line if the user selects a large scale which doesn't include all the residential roads. I'd certainly extend this to SELECT * [...] if required ;-) Done - works, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Write once. Port to many. Get the SDK and tools to simplify cross-platform app development. Create new or port existing apps to sell to consumers worldwide. Explore the Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join http://p.sf.net/sfu/intel-appdev ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
Hi Peter, what you're describing sounds familiar. Ralf and I had been observing at least two characteristic types of failures: 1.) An airfield hole (or a road) cutting a landcover polygon into two parts of which the (much) smaller one was left without centroid after clipping. 2.) A centroid sitting in a very thin edge of a polygon which resulted from clipping at small angles, in a so-called sliver whereas the sliver was later removed (cleaned) from the polygon without relocating the centroid. For the long-term I'm in favour of doing all this polygon-processing in GRASS because GRASS takes care of the topological consistency - at least as long as you don't force it not to do so. In preparation of a talk last month I did a few pictures from experiments (in PostGIS and GRASS) by creating a buffer around all the road lines - my experimental OSM line feature layer on the MapServer: http://mapserver.flightgear.org/map/?lon=6.5045lat=51.23036zoom=15layers=0BTTFTFF This turns all the various road lines into one single polygon layer, ready for clipping against the land cover: http://foxtrot.mgras.net/bitmap/FGFS/EDLN-OpenLayers-RoadCover.png which I'm then cutting out of the landcover. http://foxtrot.mgras.net/bitmap/FGFS/EDLN-OpenLayers-RoadCutout.png Images had been rendered via MapServer right from the PostGIS DB and delivered by OpenLayers. The underlying principle is very simple, indeed. Anyhow I'm still impressed that all this really works as advertized on real data ;-) Currently I'm (hopefully) running the last test-cycles for cleaning polygonal land cover at large scale in GRASS. Next step will be to verify wether v.drape yields a result which is suitable for our needs, thus by filling airport holes (via poly2ogr and some vertex elevation magic) and roads into the base land cover, just few steps remain until the stuff could be written into your favourite scenery format. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
Hi Martin, I saw the experimental polygons on the mapserver. It's good to see clipping the road against the landclass works so well. I had played with v.buffer before, and was going to use it as a baseline to try to retain texture coordinates in GRASS in the same way I am for ogr-decode / construct. So the single road poly would still be used for clipping, but when triangulating, I would still have individual road polys and column data for the texture parameters. Have you tried to triangulate the landclass in GRASS yet? I was wondering how well that works compared to triangleJRS. I ran a full xdist=2, ydist=2 fgfs-construct last night with my original idea (turns out the time consumed was caused by all the converting of point data to use clipper - reverting to GPC brings the speed back up to the original fgfs-construct). To keep triangleJRS from crashing, I'm issuing the -X flag (suppress exact arithmetic). The author says this isn't recommended, and may cause artifacts, I'll have to inspect it. http://www.cs.cmu.edu/~quake/triangle.exact.html I want to finish my fgfs-construct rework before looking into GRASS modules again, but I was wondering if we have documentation / sourcecode on how the OSM data was imported / converted to shapefiles on the mapserver. I was hoping to add a couple columns from the original XML (number of lanes / z-order). On Mon, Dec 19, 2011 at 7:45 AM, Martin Spott martin.sp...@mgras.netwrote: Hi Peter, what you're describing sounds familiar. Ralf and I had been observing at least two characteristic types of failures: 1.) An airfield hole (or a road) cutting a landcover polygon into two parts of which the (much) smaller one was left without centroid after clipping. 2.) A centroid sitting in a very thin edge of a polygon which resulted from clipping at small angles, in a so-called sliver whereas the sliver was later removed (cleaned) from the polygon without relocating the centroid. For the long-term I'm in favour of doing all this polygon-processing in GRASS because GRASS takes care of the topological consistency - at least as long as you don't force it not to do so. In preparation of a talk last month I did a few pictures from experiments (in PostGIS and GRASS) by creating a buffer around all the road lines - my experimental OSM line feature layer on the MapServer: http://mapserver.flightgear.org/map/?lon=6.5045lat=51.23036zoom=15layers=0BTTFTFF This turns all the various road lines into one single polygon layer, ready for clipping against the land cover: http://foxtrot.mgras.net/bitmap/FGFS/EDLN-OpenLayers-RoadCover.png which I'm then cutting out of the landcover. http://foxtrot.mgras.net/bitmap/FGFS/EDLN-OpenLayers-RoadCutout.png Images had been rendered via MapServer right from the PostGIS DB and delivered by OpenLayers. The underlying principle is very simple, indeed. Anyhow I'm still impressed that all this really works as advertized on real data ;-) Currently I'm (hopefully) running the last test-cycles for cleaning polygonal land cover at large scale in GRASS. Next step will be to verify wether v.drape yields a result which is suitable for our needs, thus by filling airport holes (via poly2ogr and some vertex elevation magic) and roads into the base land cover, just few steps remain until the stuff could be written into your favourite scenery format. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom
Hi Peter,
Peter Sadrozinski wrote:
Have you tried to triangulate the landclass in GRASS yet? I was wondering
how well that works compared to triangleJRS.
I have to admit that I never did that myself.
At one of our meetings maybe more than two years ago Ralf presented to
me what later ended up as v.contri GRASS module in terragear-cs.
It is supposed to be an adaption of exactly the same triangulation as
in fgfs-construct. He was able to prove successful triangulation of
a slightly complex airfield layout (something like EDDK), thus we may
assume that it's functional at least in general, but I think he never
threw large maps at it.
http://www.cs.cmu.edu/~quake/triangle.exact.html
from a certain perspective, this looks like opening yet another
can of worms ;-)
I want to finish my fgfs-construct rework before looking into GRASS modules
again, but I was wondering if we have documentation / sourcecode on how the
OSM data was imported / converted to shapefiles on the mapserver.
I've been using osm2pgsql for years for importing The Planet Dump
into PostGIS and so far I'm quite satisfied. After the bare import I'm
using a custom script to separate the various road types into distinct
tables/layers, something like:
landcover= SELECT osm_id, access, admin_level, z_order, way_area, wkb_geometry
landcover- INTO osm_motorway
landcover- FROM planet_osm_line
landcover- WHERE highway LIKE 'motorway%'
landcover- AND ST_GeometryType(wkb_geometry) LIKE 'ST_LineString';
That way I don't have to query the entire planet_osm_line if the user
selects a large scale which doesn't include all the residential roads.
I'd certainly extend this to SELECT * [...] if required ;-)
Well, and for the Scenery building I'm just using this script - see
the bottom for ${ROAD_TYPE} = OSM:
http://mapserver.flightgear.org/git/gitweb.pl?p=terragear-cs;a=blob;f=src/Prep/OGRDecode/process.sh
No Shapefiles at work here.
Cheers,
Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--
--
Learn Windows Azure Live! Tuesday, Dec 13, 2011
Microsoft is holding a special Learn Windows Azure training event for
developers. It will provide a great way to learn Windows Azure and what it
provides. You can attend the event by watching it streamed LIVE online.
Learn more at http://p.sf.net/sfu/ms-windowsazure
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom Scenery branch,
Hi Martin, I think I see the cause. It looks like sometimes, when a low priority landclass is clipped against the already accumulated thin polys like roads and steams, there could be a section that isn't removed (so only 1 region is created when there should be 2). Because the road region is still added to the triangulation, and crosses the lower priority landclass, the inside point only works for one of the regions. I showed a bit how this looks in qgis on the forum. the picture shows that a 'hole' contour actually separates the two sides of the road. of course, calc_points_inside only generates 1 point per non-hole contour, so only 1 resulting regions will have the material attribute. Cleaning up the polys after clipping, and before triangulation seems to help, but of course will not be 100% effective. Pete On Sat, Dec 17, 2011 at 9:41 PM, Martin Spott martin.sp...@mgras.netwrote: Hi Peter, Flightgear-commitlogs wrote: It doesn't crash, but sometimes, polygons lose their material. I think fixing that should be easier than the endless massaging of data to keep good looking data. According to my/our (Ralf's and mine) experience loosing the material type was usually caused by the point-in-polygon droppping the centroid into the wrong polygon. The terragear-cs history contains a 'robust' point-in-polygon routine by Ralf Gerlich which might me worth investigating. But there's a price to pay because the mentioned method might drop the centroid into areas of the polygon which later are getting eliminated without topologically correct handling of the centroid. To put it into different words: If polygon massaging occurs in a topologically safe manner, there's a robust point-in-polygon available. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] [Flightgear-commitlogs] TerraGear / Custom Scenery branch,
Hi Peter, Flightgear-commitlogs wrote: It doesn't crash, but sometimes, polygons lose their material. I think fixing that should be easier than the endless massaging of data to keep good looking data. According to my/our (Ralf's and mine) experience loosing the material type was usually caused by the point-in-polygon droppping the centroid into the wrong polygon. The terragear-cs history contains a 'robust' point-in-polygon routine by Ralf Gerlich which might me worth investigating. But there's a price to pay because the mentioned method might drop the centroid into areas of the polygon which later are getting eliminated without topologically correct handling of the centroid. To put it into different words: If polygon massaging occurs in a topologically safe manner, there's a robust point-in-polygon available. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
