Re: [Talk-GB] Quarterly Project : Schools - Wales data
On 19/01/16 15:55, Bogus Zaba wrote: >>> Is there a way of downloading the whole of the edubase database so that >>> >> I do not run into the problems of searching on terms that I may not get >>> >> quite right? >> > http://www.education.gov.uk/edubase/edubasealldata20160119.csv >> > 36Mb, and a little slow downloading, but has all the data and seems to >> > be updated regularly even if it still has old website url's and other >> > content ;) >> > > Lester - > > Thanks for this, but I think this link is not fully functional right > now. Firefox, Chromium and a simple > wget command all give me a truncated file about 6.6MB. Needless to say > it's missing the > Welsh Council areas. > > I'll try again tomorrow. I guess it's updated daily with the new date > put into the filename / URL? I just download with Firefox, opened in LibreOffice and filtered the Worcs/WR* entries to give me a working list ;) Nice to not have to jump through hoops to get a list. If you get stuck trying again, shout and I'll drop you a welsh filtered version. -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Quarterly Project : Schools - Wales data
On 19/01/16 15:25, Robert Whittaker (OSM lists) wrote: > On 19 January 2016 at 13:17, Bogus Zaba wrote: >> At the end of the day however, if Welsh schools are tagged with >> ref:estyn=* but not with ref:edubase=*, am I right in thinking that they >> will not be picked up as "progress" in the quarterly project? > All UK ways and relations tagged with either amenity=school or > amenity=college will get picked up by my tool at > http://robert.mathmos.net/osm/schools/ . I then attempt to match these > OSM objects with the schools on the official lists. Any ref:edubase, > ref:seedcode and ref:deniirn values are used on a first pass, but > after that there is a second pass that matches each unmatched school > on the official list to the nearest OSM object within 1km that hasn't > yet been matched. In rural areas (schools generally more than 1km > apart) this works very well. In urban areas the tool can often match > things the wrong way. But a match will still correspond to a school in > OSM being counted, even if it's an incorrect match. > > So apart from some of the matches in the tool being wrong, almost > everything should still be counted. (The exception is if there's a > missing postcode in the official data, or the actual location of the > school is more than 1km from the postcode centroid. Then the match > won't happen.) > > Hope that helps, > > Robert. > OK, I get that now. I'll crack on with what I was doing last week. Main problems I am seeing is with schools apparently changing names (is it just a name change?) or adding new school on the same site as houses another one. Some will require ground surveys which I will not be able to do as I draw circles of greater and greater circumference from home base. Bogus -- Dr Bogumil N Zaba ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Quarterly Project : Schools - Wales data
On 19/01/2016 15:55, Bogus Zaba wrote: On 19/01/16 14:58, Lester Caine wrote: http://www.education.gov.uk/edubase/edubasealldata20160119.csv 36Mb, and a little slow downloading, but has all the data and seems to be updated regularly even if it still has old website url's and other content ;) Lester - Thanks for this, but I think this link is not fully functional right now. Firefox, Chromium and a simple wget command all give me a truncated file about 6.6MB. Needless to say it's missing the Welsh Council areas. I just downloaded it using Leechget: 36 MB, 43962 rows in Excel. -- Steve --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Quarterly Project : Schools - Wales data
On 19/01/16 14:58, Lester Caine wrote: > On 19/01/16 13:17, Bogus Zaba wrote: >> Is there a way of downloading the whole of the edubase database so that >> I do not run into the problems of searching on terms that I may not get >> quite right? > http://www.education.gov.uk/edubase/edubasealldata20160119.csv > 36Mb, and a little slow downloading, but has all the data and seems to > be updated regularly even if it still has old website url's and other > content ;) > Lester - Thanks for this, but I think this link is not fully functional right now. Firefox, Chromium and a simple wget command all give me a truncated file about 6.6MB. Needless to say it's missing the Welsh Council areas. I'll try again tomorrow. I guess it's updated daily with the new date put into the filename / URL? Bogus Zaba -- Dr Bogumil N Zaba ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Quarterly Project : Schools - Wales data
On 19 January 2016 at 13:17, Bogus Zaba wrote: > At the end of the day however, if Welsh schools are tagged with > ref:estyn=* but not with ref:edubase=*, am I right in thinking that they > will not be picked up as "progress" in the quarterly project? All UK ways and relations tagged with either amenity=school or amenity=college will get picked up by my tool at http://robert.mathmos.net/osm/schools/ . I then attempt to match these OSM objects with the schools on the official lists. Any ref:edubase, ref:seedcode and ref:deniirn values are used on a first pass, but after that there is a second pass that matches each unmatched school on the official list to the nearest OSM object within 1km that hasn't yet been matched. In rural areas (schools generally more than 1km apart) this works very well. In urban areas the tool can often match things the wrong way. But a match will still correspond to a school in OSM being counted, even if it's an incorrect match. So apart from some of the matches in the tool being wrong, almost everything should still be counted. (The exception is if there's a missing postcode in the official data, or the actual location of the school is more than 1km from the postcode centroid. Then the match won't happen.) Hope that helps, Robert. -- Robert Whittaker ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Quarterly Project : Schools - Wales data
On 19/01/16 13:17, Bogus Zaba wrote: > Is there a way of downloading the whole of the edubase database so that > I do not run into the problems of searching on terms that I may not get > quite right? http://www.education.gov.uk/edubase/edubasealldata20160119.csv 36Mb, and a little slow downloading, but has all the data and seems to be updated regularly even if it still has old website url's and other content ;) -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Quarterly Project : Schools - Wales data
On 18/01/16 18:56, Robert Whittaker (OSM lists) wrote: > On 18 January 2016 at 16:41, Bogus Zaba wrote: >> I was however a bit surprised to see that the progress tool referenced >> here : http://robert.mathmos.net/osm/schools/progress/ >> includes Welsh postcodes but uses Edubase rather than estyn as the data >> source. As far as I can make out, estyn is the more comprehensive data >> source for Wales, Edubase is by no means complete. > I set up http://robert.mathmos.net/osm/schools/progress/ . I was aware > of the Estyn source, but was hoping that Edubase already had all the > Welsh schools covered, as the Edubase data was easier to import > automatically. You seem to be suggesting that this isn't the case. It > looks like the Estyn numbers are actually a three-digit LA code > followed by a four-digit school code, which correspond to the > "LA/Establishment Number" stored in Edubase. I did a quick check with > the Isle of Anglesey schools, and it seems both Estyn and Edubase have > the same number -- 56. If you area aware of significant numbers of > schools missing from Edubase, then please let me know, and I'll see > how easy it would be to import the Estyn data too. In the mean time, > I'm afraid the Estyn codes won't be picked up by my tool. > > Robert. > Robert - Many thanks for prompt reply. I think on further investigation you are correct - Edubase *does* contain all the schools that I have looked at in this area based on the Estyn listing. I had a full list of Estyn-listed establishments and was cross-checking to the Edubase list but using only the web-based edubase query tool (http://www.education.gov.uk/edubase/home.xhtml). I now see that in using that query you have to be careful to try both "Ruthin" and "Ruthun" or "Denbigh" and "Dinbych" - ie English and Welsh versions of the placenames. Something you might think a local would not take too long to work out... Is there a way of downloading the whole of the edubase database so that I do not run into the problems of searching on terms that I may not get quite right? Thanks also for explaining the relationship between the estyn id and the edubase establishment - that helps in tying things together. At the end of the day however, if Welsh schools are tagged with ref:estyn=* but not with ref:edubase=*, am I right in thinking that they will not be picked up as "progress" in the quarterly project? Thanks again for your work and specific help on this. Bogus Zaba -- Dr Bogumil N Zaba ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] Quarterly Project : Schools - Wales data
On 18 January 2016 at 16:41, Bogus Zaba wrote: > I was however a bit surprised to see that the progress tool referenced > here : http://robert.mathmos.net/osm/schools/progress/ > includes Welsh postcodes but uses Edubase rather than estyn as the data > source. As far as I can make out, estyn is the more comprehensive data > source for Wales, Edubase is by no means complete. I set up http://robert.mathmos.net/osm/schools/progress/ . I was aware of the Estyn source, but was hoping that Edubase already had all the Welsh schools covered, as the Edubase data was easier to import automatically. You seem to be suggesting that this isn't the case. It looks like the Estyn numbers are actually a three-digit LA code followed by a four-digit school code, which correspond to the "LA/Establishment Number" stored in Edubase. I did a quick check with the Isle of Anglesey schools, and it seems both Estyn and Edubase have the same number -- 56. If you area aware of significant numbers of schools missing from Edubase, then please let me know, and I'll see how easy it would be to import the Estyn data too. In the mean time, I'm afraid the Estyn codes won't be picked up by my tool. Robert. -- Robert Whittaker ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
[Talk-GB] Quarterly Project : Schools - Wales data
Following the wiki guidelines ( http://wiki.openstreetmap.org/wiki/UK_Quarterly_Projects ) I have been adding a ref:estyn=* tag to schools which are local to me. I plan to cover some rather further afield if other mappers in Wales do not seem to be participating enthusiastically. I was however a bit surprised to see that the progress tool referenced here : http://robert.mathmos.net/osm/schools/progress/ includes Welsh postcodes but uses Edubase rather than estyn as the data source. As far as I can make out, estyn is the more comprehensive data source for Wales, Edubase is by no means complete. Would I be correct in assuming that the progress tool is not really something that we should be relying on when it comes to looking at Welsh schools? My own overpass turbo queries suggest that there are 1474 amenity=school tags in Wales of which 36 so far have a ref:estyn=* tag. I guess we have a little way to go... bogzab -- Dr Bogumil N Zaba ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
