Re: [R-sig-Geo] sp::disaggregate() does not seem to add the necessary "comment" attribute to identify holes if Polygons does not contain exactly 1 polygon with 0 or 1 holes
Okay, I think I've tracked it down. This is... absurdly complicated and I don't know what the actual correct resolution is, but I'm pretty sure that at least I know what's going wrong. Basically it comes down to the fact that createSPComment() does not actually check whether each individual polygon in the SpatialPolygons object has a comment (much less a correct comment). Instead, it looks at a top-level "comment" attribute. If that top-level comment attribute (i.e. sf@comment for my SpatialPolygonsDataFrame object stored as "sf") is "TRUE", then createSPComment() does nothing. And here's where the second layer of the problem comes in for my dataset --- my original data, pre-disaggregation, has complete and correct comments. Some of those comments (for objects where it's a single polygon with 0 or 1 holes) are being retained by the basic disaggregation algorithm, while others are not. Your patch attempts to fill in the holes by running createSPComment() on the final results. But... the thing you are running createSPComment() on is SpatialPolygons(p), where p is the list of disaggregated polygons. And the SpatialPolygons() constructor looks at the list of the polygons you feed it, and if *any* of those polygons have comments, it sets the top-level "comment" attribute of its output to "TRUE", even if other polygons are missing comments. (And it also doesn't check the comments it gets for correctness.) So the fact that some polygon comments are retained in p means that SpatialPolygons() lies to createSPComment() about whether it has any missing comments that need to be fixed. On the other hand, when I manually run createPolygonComment() on the individual Polygons objects in disaggregate()'s output, I don't even look at the top-level "comment" attribute, so it works just fine. My little toy test polygons, on the other hand, were just not complex enough to end up with partial comments in them, so they also didn't experience this error. I am not familiar enough with the design philosophy behind sp to suggest at what level this issue should be fixed, but I hope this is at least enough info to help those who do know what they're doing get it corrected. - Anne On 04/30/2017 12:18 PM, Anne C. Hanna wrote: > Roger, > > Unfortunately I have a use case for you of createSPComment() not working. > > I tried your new version of disaggregate() on the original test script I sent > you, and it does seem to have fixed all the cases in there. However, when I > tried it on my actual data, it had the same failure mode as before. The > original dataset has what appear to be correct comments, but the > disaggregate() output is full of NULLs in place of comments for the multi-part > and multi-hole polygons. > > createSPComment() appears to be what's failing, rather than just some logic in > your disaggregate() function --- when I try to run createSPComment() on the > disaggregate() output, I still get the same set of NULLs. I can fix it if I > run createPolygonsComment() individually on every Polygons object in my > SpatialPolygonsDataFrame. > > I'm not sure what is different between my dataset and the test polygons I was > using earlier, as createSPComment() seems to handle the test shapes just fine > (and converting my SpatialPolygonsDataFrame into just a set of SpatialPolygons > also doesn't help). But presumably it is better to have createSPComment() > fixed than to have to work around it in disaggregate(). So I guess I'll look > at the code for that and see if I can find what's wrong. > > Just in case you want to see this in action, I've attached a little test > script. The data set I am working with is available at: > > http://aws.redistricting.state.pa.us/Redistricting/Resources/GISData/2011-Voting-District-Boundary-Shapefiles.zip > > - Anne > > > On 04/29/2017 10:06 AM, Roger Bivand wrote: >> On Sat, 29 Apr 2017, Anne C. Hanna wrote: >> >>> Roger, >>> >>> This looks great, and I will try it out ASAP. I do have one reservation >>> though --- it seems you are using createSPComment() to reconstruct the >>> comments, and I have seen some discussion that that may not be reliable in >>> all cases (e.g. if the initial polygons are wonky in some way). I don't >>> know a lot about this, but is it possible that it would be preferable to >>> parse and appropriately disaggregate the original comments strings (if they >>> exist), so as to deal slightly more smoothly with such cases (i.e., the >>> polygons would still be wonky, but at least the hole/polygon matching would >>> track with whatever was in the original data)? >> >> Contributions very welcome - this was code moved from the raster package, so >> I >> really have little feeling for why it is necessary. >> >> Please provide references and use cases. GEOS is strict in its treatments of >> geometries, but does work in most cases. People just expressing opinions >> doesn't help, and may well be misleading. It is possible to clean geometries >> too
Re: [R-sig-Geo] sp::disaggregate() does not seem to add the necessary "comment" attribute to identify holes if Polygons does not contain exactly 1 polygon with 0 or 1 holes
Roger, Unfortunately I have a use case for you of createSPComment() not working. I tried your new version of disaggregate() on the original test script I sent you, and it does seem to have fixed all the cases in there. However, when I tried it on my actual data, it had the same failure mode as before. The original dataset has what appear to be correct comments, but the disaggregate() output is full of NULLs in place of comments for the multi-part and multi-hole polygons. createSPComment() appears to be what's failing, rather than just some logic in your disaggregate() function --- when I try to run createSPComment() on the disaggregate() output, I still get the same set of NULLs. I can fix it if I run createPolygonsComment() individually on every Polygons object in my SpatialPolygonsDataFrame. I'm not sure what is different between my dataset and the test polygons I was using earlier, as createSPComment() seems to handle the test shapes just fine (and converting my SpatialPolygonsDataFrame into just a set of SpatialPolygons also doesn't help). But presumably it is better to have createSPComment() fixed than to have to work around it in disaggregate(). So I guess I'll look at the code for that and see if I can find what's wrong. Just in case you want to see this in action, I've attached a little test script. The data set I am working with is available at: http://aws.redistricting.state.pa.us/Redistricting/Resources/GISData/2011-Voting-District-Boundary-Shapefiles.zip - Anne On 04/29/2017 10:06 AM, Roger Bivand wrote: > On Sat, 29 Apr 2017, Anne C. Hanna wrote: > >> Roger, >> >> This looks great, and I will try it out ASAP. I do have one reservation >> though --- it seems you are using createSPComment() to reconstruct the >> comments, and I have seen some discussion that that may not be reliable in >> all cases (e.g. if the initial polygons are wonky in some way). I don't >> know a lot about this, but is it possible that it would be preferable to >> parse and appropriately disaggregate the original comments strings (if they >> exist), so as to deal slightly more smoothly with such cases (i.e., the >> polygons would still be wonky, but at least the hole/polygon matching would >> track with whatever was in the original data)? > > Contributions very welcome - this was code moved from the raster package, so I > really have little feeling for why it is necessary. > > Please provide references and use cases. GEOS is strict in its treatments of > geometries, but does work in most cases. People just expressing opinions > doesn't help, and may well be misleading. It is possible to clean geometries > too, see the cleangeo package. createPolygonsComment probably isn't foolproof, > but specific reports help, because then it is possible to do something about > it. sp classes didn't use simple features when written because sf was not > widely used then - introduced in 2004, when sp was being developed. Using sf > helps, and rgeos::createPolygonsComment was a work-around from 2010 when rgeos > was written. > > Even when you use sf in the sf package, you will still run into invalid > geometries because the geometries are in fact invalid, the sf package also > uses GEOS. > > Roger > >> >> I also had no success using createSPComment to fix disaggregate()'s output >> previously, even though my polygons are perfectly non-wonky, so I am perhaps >> a little more untrusting of it than I should be. But I'll let you know how >> this version works with my data. Thanks for addressing this so quickly! >> >> - Anne >> >> >> On 04/28/2017 12:11 PM, Roger Bivand wrote: >>> I've pushed the fix to my fork: >>> >>> https://github.com/rsbivand/sp >>> >>> and created a pull request: >>> >>> https://github.com/edzer/sp/pull/28 >>> >>> Only one part of a complicated set if nested if() in disaggregate() was >>> adding >>> comments, but in some settings the existing comments survived the >>> disaggregation. Now the Polygons object comment attribute is re-created for >>> all Polygons objects. This is version 1.2-6, also including code changes >>> that >>> internally affect rgdal and rgeos - you may like to re-install them from >>> source after installing this sp version (shouldn't matter). >>> >>> Roger >>> >>> On Fri, 28 Apr 2017, Anne C. Hanna wrote: >>> Hello. I first posted this issue report on the sp GitHub repo (https://github.com/edzer/sp/issues/27) and it was suggested that I redirect it to here. I am working with a geographic dataset with complex borders. The data is stored as a SpatialPolygonsDataFrame. Each Polygons object in the data frame may be composed of multiple disjoint polygons, and each polygon may have multiple holes. I want to disaggregate each of the Polygons objects into its individual disjoint polygons and construct an adjacency matrix for all the disjoint components, and I was using disaggregate() to do this. However, wh
