[Talk-GB] TfL Cycling Infrastructure Database - conflation
Dear all, As you may recall, Transport for London (TfL) released as open data a major new cycling infrastructure dataset. Various people within the OSM UK community met TfL in the run-up to its release, and it was well-received. The OSM wiki has a project page here: https://wiki.openstreetmap.org/wiki/TfL_Cycling_Infrastructure_Database and you can browse the data here: https://bikedata.cyclestreets.net/tflcid/ I'm pleased to say that TfL, through a project with London Cycling Campaign, has allocated resources to enable conversion of the data as well as officer time within TfL to help conflate what is a huge dataset. CycleStreets is working with Richard Fairhurst (cycle.travel, and of course well-known as a long-standing member of the community) to get this conversion work done. Richard will be doing the bulk of the scripting work, and is working on converting each of the sections of data. This will naturally be published on Github openly, as will the outputted data. This is reasonably complex work given the number of attributes and the data extent. We are keen to ensure the OSM community is able to scrutinise the conversion easily and have input. Richard will post to this list about the work, as it proceeds. We will be using the previously-discussed conversion table: https://bikedata.cyclestreets.net/tflcid/conversion/ https://github.com/cyclestreets/tflcid-conversion and the remaining issues will be mopped up during the work. The aim of the scripting is to get as much of the data conversion automated as possible, and matching of assets very reliable, so that the conflation (tool yet to be determined) can then be done with a high degree of confidence and as easily as possible. The conflation itself, using the output of the script, will be started by TfL personnel, with training from Richard/myself about both process and norms and quality expectations of the community. TfL only have a certain amount of time resource for this, so it is hoped the OSM community will also contribute time as we refine and document the process. As noted above, the converted data will be published along the script itself. Every asset also has two images (already publicly available) which will be useful for verification. Richard and I hope this news will be well-received within the OSM community - this is a great opportunity to enhance OSM data in London. For instance, cycle parking coverage and detail can be considerably enhanced as a result of this data. If you have feedback for TfL on the CID outside of this conflation task then they can be contacted via c...@tfl.gov.uk. Martin, ** CycleStreets - For Cyclists, By Cyclists Developer, CycleStreets ** https://www.cyclestreets.net/ ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] TfL Cycling Infrastructure Database
On Tue, 11 Jun 2019, Tom Hughes wrote: Whether we want it is not really the issue. I believe the issue is the licensing, and until that is resolved what we may or may not want is irrelevant. I have been following up with TfL some of the licensing questions, and am expecting very shortly to send updated information I have from TfL to the Licensing Working Group[1]. I'm awaiting one or two further clarifications before I send that. [1] https://wiki.osmfoundation.org/wiki/Licensing_Working_Group Martin, ** CycleStreets - For Cyclists, By Cyclists Developer, CycleStreets ** https://www.cyclestreets.net/ ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] TfL Cycling Infrastructure Database
On Tue, 11 Jun 2019, Tony Shield wrote: Looking at the demo I can't think of a reason why OSM would not want this data - I believe we do want this data. Questions I have - * Are the tags suitable for a global database? Can and should they be reused elsewhere in UK? or globally? Is there a need for specific prefixed TfL tags? I'm about to circulate a proposed mapping of TfL's attributes to OSM tags. In some cases, there is no equivalent, e.g. signage plates/locations do not have an equivalent in OSM). In some cases I believe there are new tags that could be created, e.g. there are a lot of cyclehoop stands which are just arms bolted to an existing signpost, and which there doesn't seem to be an existing OSM tag for (surprisingly). I certainly don't believe that OSM should be creating anything London-specific or maintaining proprietary prefixing. * Are there clashes with other cycling data in London? What is a resolution strategy? * Will import and integration be performed by automatic processing? Usual OSM import rules? Clash detection and sanity checking? Clearly, there is overlap with existing OSM data, in that for instance, a high proportion of the cycle/bus lanes are present, cycle parking is sometimes present (though the TfL data seems more comprehensive to me), etc. TfL seem potentially interested in funding tool development, which in my view should be of a generic nature that supports other conflation efforts. Richard Fairhurst's new tool, and other things like the JOSM conflation tool is amongst others I am currently looking at. I would welcome any pointers to other tools that people would recommend or are aware of. It seems to me that a manual conflation process, assisted by pre-processing to weed out obvious clashes or mismatches, and with workflows optimised towards the large volume of data here (e.g. approving several nearby cycle parking locations in one go) would be the way forward. Again, any views most welcome. Martin, ** CycleStreets - For Cyclists, By Cyclists Developer, CycleStreets ** https://www.cyclestreets.net/ ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] TfL Cycling Infrastructure Database
The demonstrator map showing this data from TfL now has an extensive set of filters for each feature type: https://tflcid.cyclestreets.net/ E.g. for cycle lane/track, you can now filter for segregated, shared, prority at sideroads, full/part time, mandatory, stepped track, etc. These filters match the schema definition at: https://tflcid.cyclestreets.net/TfL_CID_Schema.pdf There are also a lot more images loaded now. My general impression is that the data does seem very accurate. I would welcome more eyes on this, and thoughts generally on the use of this data in OSM. Hopefully the filtering will make this easier to find various kinds of infrastructure. I am about to circulate a suggested mapping of the TfL data to OSM equivalents, where equivalent tags exist, for comments. NB The new filtering controls have a couple of URL persistency bugs - I'll mop these up shortly. So please be aware of this if sending a link. Martin, ** CycleStreets - For Cyclists, By Cyclists Developer, CycleStreets ** https://www.cyclestreets.net/ On Fri, 10 May 2019, Martin Lucas-Smith - CycleStreets wrote: Transport for London (TfL) have created a new database of cycling infrastructure, containing 240,000 assets, covering all of Greater London. This groundbreaking database contains every cycle infrastructure asset within Greater London, including assets on and off-carriageway. The assets surveyed are: cycle parking; signals; signage; traffic calming measures; restricted points (e.g. steps); advanced stop lines; crossings; cycle lanes/tracks; and restricted routes (e.g. pedestrian only routes). TfL is keen to make this available to the OpenStreetMap community under a compatible open license, to ensure maximum use of the CID. TfL is also potentially willing to consider tool development to help facilitate sensitive merging in of this data. There is a new Wiki page, giving full details, at: https://wiki.openstreetmap.org/wiki/TfL_Cycling_Infrastructure_Database Demonstrator map: - A demonstrator map, for the purposes only of evaluation by the OSM community at this stage, has been created by CycleStreets. This demonstrator map contains only one of the 25 areas that have been surveyed. We are specifically seeking comments on data quality and usefulness of this data from the OSM community. Initial analysis by CycleStreets is that the data is of excellent quality, and very suitable for conflation into OSM, to increase both comprehensiveness and metadata quality. https://tflcid.cyclestreets.net/ (Use the controls on the right to change feature type.) Usage notes: The controls on the right of the map allow the different feature types to be selected. The OSM layer (available at zoom level 19+) also provides a live feed from the OSM API, to enable quick comparisons. The two photos of each asset are in the process of being supplied; those already available and cleared in GDPR terms are included in the popup. It is stressed that at this point, no permission is given for re-use of the data in any way, but TfL strongly intends to make this available in future. All 25 areas would be covered in the final data release, not merely the one shown currently in the demonstrator map. Feedback is very strongly encouraged, as soon as possible. What are people's thoughts? Martin, ** CycleStreets - For Cyclists, By Cyclists Developer, CycleStreets ** https://www.cyclestreets.net/ ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] TfL Cycling Infrastructure Database
On Fri, 10 May 2019, Jez Nicholson wrote: Their data is highly accurate, Yes, that seems to me as well to be the case. We're just awaiting more images to be uploaded to the site (every feature has two images, but not all are GDPR-cleared yet). I'd welcome as many eyes as possible on the sample data to get a good assessment of the data quality. https://tflcid.cyclestreets.net/ and there's definitely going to need to be some clever conflation tooling. Bike stands are fine, but advance stop lines, etc. are specialist subjects in my book. I'm sightly overawed by the quantity and am unsure whether volunteers are going to be able to get through it, but again that is something you'll be talking about in your report, no? Yes, that will be a key issue. Bear in mind that the sample data is only one of *25* areas, so there's a lot of data. Clearly, pre-translations in the data to convert the CID schema to OSM tagging would remove a lot of manual work, and a conflation tool could work on a similar basis to the England Cycling Data project tool [1]. I think there's scope for some pre-processing (e.g. eliminating locations in the CID data that clearly already exist in OSM based on a nearness search), and the ability for multiple features to be done at once, e.g. a screen where say 10-20 cycle parking locations could be reviewed at once. Again, views on this would be extremely welcome. There would need to be some tool development regardless of who does the conflation. Indeed. I'd welcome pointers to up-to-date information on the state of such tools at the moment, e.g. the JOSM tool, and other developments currently happening. [1] See images on: https://wiki.openstreetmap.org/wiki/England_Cycling_Data_project Martin, ** CycleStreets - For Cyclists, By Cyclists Developer, CycleStreets ** https://www.cyclestreets.net/ ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
[Talk-GB] TfL Cycling Infrastructure Database
Transport for London (TfL) have created a new database of cycling infrastructure, containing 240,000 assets, covering all of Greater London. This groundbreaking database contains every cycle infrastructure asset within Greater London, including assets on and off-carriageway. The assets surveyed are: cycle parking; signals; signage; traffic calming measures; restricted points (e.g. steps); advanced stop lines; crossings; cycle lanes/tracks; and restricted routes (e.g. pedestrian only routes). TfL is keen to make this available to the OpenStreetMap community under a compatible open license, to ensure maximum use of the CID. TfL is also potentially willing to consider tool development to help facilitate sensitive merging in of this data. There is a new Wiki page, giving full details, at: https://wiki.openstreetmap.org/wiki/TfL_Cycling_Infrastructure_Database Demonstrator map: - A demonstrator map, for the purposes only of evaluation by the OSM community at this stage, has been created by CycleStreets. This demonstrator map contains only one of the 25 areas that have been surveyed. We are specifically seeking comments on data quality and usefulness of this data from the OSM community. Initial analysis by CycleStreets is that the data is of excellent quality, and very suitable for conflation into OSM, to increase both comprehensiveness and metadata quality. https://tflcid.cyclestreets.net/ (Use the controls on the right to change feature type.) Usage notes: The controls on the right of the map allow the different feature types to be selected. The OSM layer (available at zoom level 19+) also provides a live feed from the OSM API, to enable quick comparisons. The two photos of each asset are in the process of being supplied; those already available and cleared in GDPR terms are included in the popup. It is stressed that at this point, no permission is given for re-use of the data in any way, but TfL strongly intends to make this available in future. All 25 areas would be covered in the final data release, not merely the one shown currently in the demonstrator map. Feedback is very strongly encouraged, as soon as possible. What are people's thoughts? Martin, ** CycleStreets - For Cyclists, By Cyclists Developer, CycleStreets ** https://www.cyclestreets.net/ ___ Talk-GB mailing list Talk-GB@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-gb
Re: [Talk-GB] UK Bike Shop dataset obtained: please merge in locations in your area
On Tue, 14 Sep 2010, Martin Lucas-Smith - CycleStreets wrote: http://www.cyclestreets.net/blog/2010/09/10/get-all-uk-bike-shops-in-osm/ Andy Allan and Shaun McDonald have created a webapp for the specific purpose of merging (manually) this data into OSM. http://shaunmcdonald.dev.openstreetmap.org/bike-shop-locator/ Effectively it has a map of locations not yet reconciled, i.e. a checklist. Click on each location, and you'll then have a direct link to the relevant bit of OSM where the shop can be added/updated/removed using personal knowledge. Once that's done, it can be ticked off back in the webapp. The new Bike Hub iPhone app, which includes a bike shop finder has just gone live*, so there is even more reason now to get the remaining locations into OSM. At present the data received by the app is coming from a local copy of the ACT data held on the CycleStreets server and exposed through our places.xml API. However, I am very keen to reroute this internally to using OSM proper, as soon as most of the shops are in OSM. So please do go to: http://shaunmcdonald.dev.openstreetmap.org/bike-shop-locator/ and merge in the remaining data or check off ones that are already in OSM. Shaun has made some further usability improvements to the webapp which makes the steps required clearer. Martin * See this guest post: http://www.cyclestreets.net/blog/2010/09/20/bike-hub-app/ (our own CycleStreets app is imminent, by the way!) ___ Talk-GB mailing list Talk-GB@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-gb
[Talk-GB] UK Bike Shop dataset obtained: please merge in locations in your area
We've brokered a dataset of all 2,500-ish bike shops in the UK from the Association of Cycle Traders (many thanks to them!), for the purposes of merging this dataset into the OpenStreetMap database. It has postcode-level accuracy only but that's a massive head-start. Read more at: http://www.cyclestreets.net/blog/2010/09/10/get-all-uk-bike-shops-in-osm/ Andy Allan and Shaun McDonald have created a webapp for the specific purpose of merging (manually) this data into OSM. http://shaunmcdonald.dev.openstreetmap.org/bike-shop-locator/ Effectively it has a map of locations not yet reconciled, i.e. a checklist. Click on each location, and you'll then have a direct link to the relevant bit of OSM where the shop can be added/updated/removed using personal knowledge. Once that's done, it can be ticked off back in the webapp. Martin, ** CycleStreets - For Cyclists, By Cyclists Developer, CycleStreets ** http://www.cyclestreets.net/ ___ Talk-GB mailing list Talk-GB@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-gb
