Hi Dan, Thanks for this! We were disappointed to find the same thing -- the correlation is just not there. I think we need to focus heavily on the improvements in our recommendation section so we can nudge these numbers towards something more reasonable, because at present it's just not good.
Best, Robert Robert Banick | Field GIS Coordinator | International Services | Ì American Red Cross <http://www.redcross.org/> 2025 E Street NW, Washington, DC 20006 Tel 202-303-5017 | Cell 202-805-3679 | Skype robert.banick On 2/16/14 4:47 PM, "Dan S" <danstowell+...@gmail.com> wrote: >Hi Robert, > >Thanks for pointing me to the raw data. I've run a crosstabulation >(matching up objects by their osm ID) and it seems to look like this: > > no damaged destroyed >no 85 82 63 >partial 170 119 111 >major 173 160 196 >total 88 31 70 > >This is rather disappointing - there is some correlation between the >two types of annotation but a lot of noise. For example the osm 'no' >category seems to be more likely to be _any_ of the other categories >in your observations rather than 'no'! > >I quantified the predictability of one from the other (using mutual >information) and it confirms this, comes out rather low. My analysis >code is at https://gist.github.com/danstowell/9040956 - just a quick >evening script, check it before relying on it... > >Best >Dan > > >2014-02-14 14:53 GMT+00:00 Banick, Robert <robert.ban...@redcross.org>: >> Hi Dan, >> >> There's a "download raw data" button on the side of the website at >> americanredcross.github.io/OSM-Assessment. Feel free to download and >>play >> with the data further -- and do contribute back anything new you find. >> We're running more intensive stats analysis on the data now and will be >> working that into the final report. >> >> Cheers, >> Robert >> >> Robert Banick | Field GIS Coordinator | International Services | Ì >> American Red Cross <http://www.redcross.org/> >> 2025 E Street NW, Washington, DC 20006 >> Tel 202-303-5017 | Cell 202-805-3679 | Skype robert.banick >> >> >> >> >> >> On 2/14/14 4:18 AM, "Dan S" <danstowell+...@gmail.com> wrote: >> >>>Hi Robert, >>> >>>Thanks for this. It's great to have some concrete analysis of what we >>>did, and the extent to which it's accurate. >>> >>>Is there a full cross-tabulation available of the numbers for >>>building-osm-status vs building-true-status? In the report there's a >>>table of over/underrepresentation, and some other stats, but it >>>doesn't give me a complete summary of whether our errors were in the >>>form of "bias" (e.g. consistently labelling things worse or better >>>than they are) or "variance" (e.g. the labelling tends to be a bit >>>random). I'd be really grateful if you could provide the full >>>cross-table. >>> >>>Dan >>> >>> >>>2014-02-12 14:22 GMT+00:00 Banick, Robert <robert.ban...@redcross.org>: >>>> Dear HOT Communuity, >>>> >>>> >>>> >>>> The American Red Cross and the REACH Initiative are pleased to present >>>>an >>>> interim assessment report on the validity of the building damages >>>>assessed >>>> through OpenStreetMap in the weeks following Typhoon Haiyan. You can >>>>find a >>>> print copy attached and a more interactive website version at the >>>>above >>>> link. >>>> >>>> >>>> The results were unfortunately negative and underline real limitations >>>>in >>>> OpenStreetMap's ability to capture these results in the present. >>>> Neverthless, this report identifies strong promise in the OSM model of >>>> crowdsourcing and highlights the investments needed to make that >>>>potential >>>> possible. It's our sincere hope that funders, NGO partners and most >>>> especially the OpenStreetMap community will rally around these >>>>investments >>>> so that OSM can play an even stronger and more operationally useful >>>>role in >>>> future disaster responses. >>>> >>>> >>>> We are indebted to the US Agency for International Development's >>>>Office >>>>of >>>> Foreign Disaster Assistance (OFDA) for funding this assessment and >>>>look >>>> forward to future partnerships to improve the utility of open data and >>>> OpenStreetMap in particular for disaster response. >>>> >>>> >>>> >>>> With all the best, >>>> >>>> Robert Banick, Dale Kunce and Clay Westrope >>>> >>>> American Red Cross & REACH Initiative >>>> >>>> >>>> Robert Banick | Field GIS Coordinator | International Services | Ì >>>>American >>>> Red Cross >>>> >>>> 2025 E Street NW, Washington, DC 20006\ >>>> >>>> >>>> _______________________________________________ >>>> HOT mailing list >>>> HOT@openstreetmap.org >>>> https://lists.openstreetmap.org/listinfo/hot >>>> >> _______________________________________________ HOT mailing list HOT@openstreetmap.org https://lists.openstreetmap.org/listinfo/hot
