Re: [osmosis-dev] Improve relation extraction handling
Brett, Brett Henderson wrote: I'm thinking of adding a cascadingRelations=yes flag to the --bp/--bb tasks that would * process relations not containing other relations as usual, * throw all other relations onto a stack and process them in a suitable order afterwards I'd try to use the existing completeXXX options as a starting point. I have done that now and committed the enhancements in r24342. I ran some tests of my own and I think everything works as desired. I'd love to amend the tests as well but it seems I am lacking either the proper system or the proper brains to run the existing tests at all. The BoundingBoxFilterTest and PolygonFilterTest work in 0.38 and they now fail in trunk; the AreaFilterTest fails in 0.38 as well as in trunk: test: [mkdir] Created dir: /home/fred/svn.openstreetmap.org/applications/utils/osmosis/tags/0.38/areafilter/report/test [junit] Running org.openstreetmap.osmosis.areafilter.v0_6.AreaFilterTest [junit] Tests run: 5, Failures: 5, Errors: 0, Time elapsed: 0.348 sec How do I find out why exactly these tests fail? Running ant with -v only gives lines like [junit] junit.framework.TestListener: startTest(testProcessBoundContainer1) [junit] junit.framework.TestListener: addError(testProcessBoundContainer1, org.openstreetmap.osmosis.areafilter.v0_6.PolygonFilter.init(Lorg/openstreetmap/osmosis/core/filter/common/IdTrackerType;Ljava/io/File;ZZZ)V) [junit] junit.framework.TestListener: addError(testProcessBoundContainer1, null) [junit] junit.framework.TestListener: endTest(testProcessBoundContainer1) Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
[osmosis-dev] Improve relation extraction handling
Hi, I'm a major user of the --bp task in Osmosis, for the nightly Geofabrik country extracts. I cannot use the complete-something flags as that would take far too much time, and on the whole the clip-incomplete-entities mode seems not to be too bad for users. There's one thing that has popped up on this list however (lost relations while clipping) which is an increasing concern to our users and which I hope I can fix. The problem is that with cascading relations, Osmosis will only include a relation if one of its children has been included already, so if node N1 is inside the clipping area, relation R1 contains node N1, relation R2 contains relation R1 then relation R1 will alway be included in the output, but relation R2 will only be included if it comes after R1 in the input file. Now while using the temporary storage for nodes and ways would be too expensive for the nightly Geofabrik build, I think that temporarily storing relations would be acceptable (as there aren't too many). I'm thinking of adding a cascadingRelations=yes flag to the --bp/--bb tasks that would * process relations not containing other relations as usual, * throw all other relations onto a stack and process them in a suitable order afterwards I am aware that determining a suitable order could be difficult if you have many levels of cascasing (or even circular) relations, and that the output file would not be ordered properly, but I think such an option would mean a big improvement at a relatively low cost. 1. Any comments? 2. Anyone done something like that already? 3. Is any of Osmosis' internal storage mechanisms suitable for buffering relations in this way or do I have to invent something new? Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Improve relation extraction handling
Hi Frederik, On Sun, Nov 7, 2010 at 10:44 PM, Frederik Ramm frede...@remote.org wrote: Hi, I'm a major user of the --bp task in Osmosis, for the nightly Geofabrik country extracts. I cannot use the complete-something flags as that would take far too much time, and on the whole the clip-incomplete-entities mode seems not to be too bad for users. There's one thing that has popped up on this list however (lost relations while clipping) which is an increasing concern to our users and which I hope I can fix. The problem is that with cascading relations, Osmosis will only include a relation if one of its children has been included already, so if node N1 is inside the clipping area, relation R1 contains node N1, relation R2 contains relation R1 then relation R1 will alway be included in the output, but relation R2 will only be included if it comes after R1 in the input file. Now while using the temporary storage for nodes and ways would be too expensive for the nightly Geofabrik build, I think that temporarily storing relations would be acceptable (as there aren't too many). I'm thinking of adding a cascadingRelations=yes flag to the --bp/--bb tasks that would * process relations not containing other relations as usual, * throw all other relations onto a stack and process them in a suitable order afterwards I am aware that determining a suitable order could be difficult if you have many levels of cascasing (or even circular) relations, and that the output file would not be ordered properly, but I think such an option would mean a big improvement at a relatively low cost. 1. Any comments? It sounds good to me. 2. Anyone done something like that already? Not exactly, although it should be very similar to the existing completeXXX functionality. 3. Is any of Osmosis' internal storage mechanisms suitable for buffering relations in this way or do I have to invent something new? I'd try to use the existing completeXXX options as a starting point. The existing file system stores should work for your purposes. If their performance is not good enough then it might be necessary to investigate other options. In that case if you did come up with a higher performance design it might also be able to be used for the completeXXX options as well. Cheers, Brett ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
