Re: [Talk-GB] Import UK postcode data?

[Mark Goodge]

On 04/10/2019 20:28, Frederik Ramm wrote:

Hi,

On 10/4/19 20:51, Mark Goodge wrote:

The reality is that people expect postcodes to be a functional
search term on online mapping, at least in the UK,


You *are* ware that UK post codes are fully findable on the OSM
website and any site that runs the Nominatim geocoder? It must have
been mentioned somewhere in this thread. This means that our web site
and anything that uses Nominatim for geocoding already knows UK post
codes without importing them to OSM.


Yes, but OSM is the data, and the data needs to stand alone without 
needing Nominatim as a front-end. It's easy to add postcode search to 
OSM by means of an add-on system. But not every use of OSM data will 
include those add-ons.



With an automated import, OSM can be as up to date as the latest
release of CPO/ONSPD. And that's a positive selling point for our
data.


The notion that automated imports could set OSM apart from the 
competition flies in the face of what many of us believe to be OSM's 
unique value proposition. We don't usually brand ourselves as "the 
database with the better imports" and we're unlikely to ever be a

match to the giants on the field of engineering.


Automating a postcode import is trivial. The reason Google is slow to do 
it isn't because it's an engineering challenge, it's simply because they 
don't consider it a priority.


Richard makes a good point (that if anything, a manual process that 
allows our human editors with local knowledge - who are what really

sets us apart - to verify and improve the data would be preferable)
but also a questionable one (in suggesting that there are 195
countries in the world having some form of post codes that is also
available as open data - the number is probably one-digit).

I would like to applaud Ken for his roll-up-sleeves approach. It 
shouldn't be too hard to find one house for each of the post codes

in your local area and add the post code to that, which will
ultimately make every post code findable without actually having to
add something as synthetic as a centroid.


There are, typically, between 1,000 and 3,000 new postcodes added every 
month, and postcodes are deleted every month, too. Sometimes, there's a 
mass reorganisation of postcode data which can involve additions and 
deletions of the many tens of thousands in one month. I don't think that 
OSM has anywhere near the number of human editors necessary to keep up 
with that manually.


Mark

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Import UK postcode data?

[Chris Hill]

On 04/10/2019 15:41, Richard Fairhurst wrote:

Twopenn'orth and not particularly a reply to any single message:

1. I'm not against them being in the OSM database, mostly for the reason
that it's unrealistic to expect every single app to do additional processing
for all 195 countries in the world. Sure, it would be nice if Osmand and
maps.me and Fred's routing app and Jo's OSM-based game were all smart enough
to ingest CodePoint Open (and its 194 equivalents worldwide), but they
won't. Expecting them to do so is akin to people expecting every single app
to filter out C-roads in Britain, and even osm-carto doesn't do that. So it
seems a reasonably pragmatic thing to do.

2. However... just blindly importing them seems to be a real missed
opportunity. If you give me a nice interface with centroids for Charlbury, I
will have a go at mapping them to actual, useful polygons, based on my
knowledge of the street layout and Carla the post-lady's daily rounds (or I
could ask her, but I'm not sure of the IP of asking an RM employee...). If
you dump them into the database as-is I almost certainly won't get round to
it.

cheers
Richard

One such map is here: 
https://oscompare.raggedred.net/?zoom=16=53.73916=-0.49208=BFF 



It uses the same overlay I provide for editors: 
https://codepoint.raggedred.net 


The OSCompare does need a tidy up and would work better as a 
Leaflet-based map, maybe that's the jib for the rainy Saturday that's 
coming.


Fill y'boots :-)

--
cheers
Chris Hill (chillly)

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Import UK postcode data?

[Chris Hill]

On 04/10/2019 01:52, nd...@redhazel.co.uk wrote:


Besides, the main reason for importing these data is that we can get 
_all_ postcodes in the database. This gives users confidence that when 
they search for a postcode they will reliably get a result they are 
looking for. This is not possible when merging postcodes with 
buildings simply because we still have only a small fraction of 
buildings in the database.


Searching for GB postcodes on the OSM site works already. The search 
system, Nominatim, has extra datasets loaded to improve the search and 
one of them already is Codepoint Open. If you search for postcode that 
is not currently in a GB address (it's not yet been added possibly 
because the buildings it describes have not been created in the database 
yet) search still find it and centres the map on the imaginary postcode 
centroid. So to make searching work we do not need to import imaginary 
postcode centroids to the main OSM database.


--
cheers
Chris Hill (chillly)


___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Import UK postcode data?

[Frederik Ramm]

Hi,

On 10/4/19 20:51, Mark Goodge wrote:
> The reality is that people expect postcodes to be a functional search term on 
> online mapping, at least in the UK,

You *are* ware that UK post codes are fully findable on the OSM website
and any site that runs the Nominatim geocoder? It must have been
mentioned somewhere in this thread. This means that our web site and
anything that uses Nominatim for geocoding already knows UK post codes
without importing them to OSM.

This discussion, therefore, is not about improving the OSM web site or
indeed most web sites that use OSM ("functional search term on online
mapping"), but only (quoting Richard) "Osmand and maps.me and Fred's
routing app and Jo's OSM-based game" insofar as these don't use
Nominatim or directly ingest the available open data.

> With an automated import, OSM can be
> as up to date as the latest release of CPO/ONSPD. And that's a positive
> selling point for our data.

The notion that automated imports could set OSM apart from the
competition flies in the face of what many of us believe to be OSM's
unique value proposition. We don't usually brand ourselves as "the
database with the better imports" and we're unlikely to ever be a match
to the giants on the field of engineering.

(Maybe you're right and Google have a glitch somehow that makes them
ingest new data with a delay but that sounds like an engineering problem
that can and will be fixed.)

Richard makes a good point (that if anything, a manual process that
allows our human editors with local knowledge - who are what really sets
us apart - to verify and improve the data would be preferable) but also
a questionable one (in suggesting that there are 195 countries in the
world having some form of post codes that is also available as open data
- the number is probably one-digit).

I would like to applaud Ken for his roll-up-sleeves approach. It
shouldn't be too hard to find one house for each of the post codes in
your local area and add the post code to that, which will ultimately
make every post code findable without actually having to add something
as synthetic as a centroid.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Import UK postcode data?

[Mark Goodge]

On 04/10/2019 01:52, nd...@redhazel.co.uk wrote:


This may not be a perfect solution but the information CPO/ONSPD 
contains is still extremely useful for geocoding. Search for a postcode 
and you are _guaranteed_ to get an address in a close vicinity to a 
place you are looking for. How about not needing to start Google Maps 
when searching for a location on the go?


I entirely agree with this. You can search for any postcode on Google 
Maps, Bing Maps, Here Maps and Michelin Maps, to name just the ones I've 
checked right now. The reality is that people expect postcodes to be a 
functional search term on online mapping, at least in the UK, and not 
having it places OSM at a distinct disadvantage compared to the 
commercial mapping operators. Given that postcode data is itself 
available under a compatible licence, refusing to use it because of 
purist notions of what a map should be is, in my opinion, simply a case 
of cutting off our noses to spite our face. It's important that OSM 
gives users what they want from a map, not just what mapping geeks want 
from a map. And users want postcodes - all of them - to be a functional 
search term.


In fact, OSM can easily do better than Google et al when it comes to 
postcodes. One of their biggest faults is the lag on adding new 
postcodes - it can take up to 18 months before new postcodes become 
usable search terms on Google Maps. With an automated import, OSM can be 
as up to date as the latest release of CPO/ONSPD. And that's a positive 
selling point for our data.


Mark

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Import UK postcode data?

[Ken Kilfedder]

Thanks for this Chris,

I've just added about 100 addr:postcodes to some of my manor in no time at all. 
It was mostly pretty easy to identify a good match for and existing building. 
Then I went back and added a source tag as an afterthought.

Great stuff all round, I''l try to remember to make at trip to raggedred.net 
part of future mapping exploits.

---
https://hdyc.neis-one.org/?spiregrain
spiregrain_...@ksglp.org.uk


On Wed, 2 Oct 2019, at 2:20 PM, Chris Hill wrote:
> I would not like to see that happen. OSM maps real objects, postcodes are not 
> real and only apply as a part of an object's address. They apply to buildings 
> (delivery points on buildings really). The postcodes in Codepoint Open are 
> centroids derived from a combination of all the delivery points that share 
> the postcode so are not at all real-world objects.

> If you want to apply postcodes to addresses you can see the map overlay I 
> have produced which you can use in editors as an overlay: 
> https://codepoint.raggedred.net/ I will update it again shortly. You can also 
> derive postcodes from other open data sources such as FHRS data. 

> -- 
cheers
Chris Hill (chillly)

> 
> On 02/10/2019 13:43, Russ Phillips via Talk-GB wrote:
>> Hi,

>> I'm wondering if it would be feasible and advisable to import the UK 
>> postcode data from OS OpenData Codepoint 
>> .

>> The licence is OSM compatible. My thinking was that we could create a node 
>> for each data point and set the addr:postcode tag. This would be useful for 
>> routing software like OsmAnd, since it would allow a user to enter a 
>> postcode as a destination.

>> I'm happy to do the work, but the import guidelines 
>>  say that imports 
>> should be discussed on the imports@ list and the appropriate local 
>> communities, hence this email.

>> Russ Phillips

> 
> ___
> Talk-GB mailing list
> Talk-GB@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-gb
> 
___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Import UK postcode data?

[Richard Fairhurst]

Twopenn'orth and not particularly a reply to any single message:

1. I'm not against them being in the OSM database, mostly for the reason
that it's unrealistic to expect every single app to do additional processing
for all 195 countries in the world. Sure, it would be nice if Osmand and
maps.me and Fred's routing app and Jo's OSM-based game were all smart enough
to ingest CodePoint Open (and its 194 equivalents worldwide), but they
won't. Expecting them to do so is akin to people expecting every single app
to filter out C-roads in Britain, and even osm-carto doesn't do that. So it
seems a reasonably pragmatic thing to do.

2. However... just blindly importing them seems to be a real missed
opportunity. If you give me a nice interface with centroids for Charlbury, I
will have a go at mapping them to actual, useful polygons, based on my
knowledge of the street layout and Carla the post-lady's daily rounds (or I
could ask her, but I'm not sure of the IP of asking an RM employee...). If
you dump them into the database as-is I almost certainly won't get round to
it.

cheers
Richard



--
Sent from: http://gis.19327.n8.nabble.com/Great-Britain-f5372682.html

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Import UK postcode data?

[Dan S]

Op vr 4 okt. 2019 om 14:07 schreef Russ Garrett :
>
> On Fri, 4 Oct 2019 at 13:59, David Woolley  wrote:
> > Although I don't have a primary source for this, my understanding is
> > that the median is snapped to the nearest actual delivery point within
> > the postcode.
>
> I was also under the impression that they were mathematical centroids
> of the postcode area but the user guide [1] indeed says: "The point is
> given the coordinates of the nearest delivery point to the calculated
> mean position of the delivery points within the postcode unit."
> (Although this may not be true for all entries - the PQI will provide
> that info.)

FWIW the technical term is "medoid"... though I guess throwing new
technical terms in is not my best contribution...


> So I guess it would technically be possible to use Code-Point Open to
> add at least one address location to OSM for most postcodes. I'm still
> uneasy about doing this automatically though.

I've long been frustrated by the uselessness of most OSM-based
services wrt UK postcodes, and I agree with the argument that postcode
medoids are not very different from many of the other "abstract-ish"
things we have deliberately tagged in OSM
 - so, for me, I would support some kind of perhaps semi-automatic
import process.

Best
Dan


> [1] 
> https://www.ordnancesurvey.co.uk/documents/product-support/user-guide/code-point-open-user-guide.pdf
>
> --
> Russ Garrett
> r...@garrett.co.uk
>
> ___
> Talk-GB mailing list
> Talk-GB@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-gb

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Import UK postcode data?

[Russ Garrett]

On Fri, 4 Oct 2019 at 13:59, David Woolley  wrote:
> Although I don't have a primary source for this, my understanding is
> that the median is snapped to the nearest actual delivery point within
> the postcode.

I was also under the impression that they were mathematical centroids
of the postcode area but the user guide [1] indeed says: "The point is
given the coordinates of the nearest delivery point to the calculated
mean position of the delivery points within the postcode unit."
(Although this may not be true for all entries - the PQI will provide
that info.)

So I guess it would technically be possible to use Code-Point Open to
add at least one address location to OSM for most postcodes. I'm still
uneasy about doing this automatically though.

Cheers,

[1] 
https://www.ordnancesurvey.co.uk/documents/product-support/user-guide/code-point-open-user-guide.pdf

-- 
Russ Garrett
r...@garrett.co.uk

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Import UK postcode data?

[David Woolley]

On 04/10/2019 13:47, Dave F via Talk-GB wrote:
No. The centre point is not associated with *any* delivery point. It is 
an arbitrary mean, calculated mathematically. it could, in theory, be 
located in the middle of a park.
Even postcodes unique to one property/business aren't accurate as their 
positions are misaligned by the effect of adjacent areas.


Although I don't have a primary source for this, my understanding is 
that the median is snapped to the nearest actual delivery point within 
the postcode.


___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Import UK postcode data?

[Dave F via Talk-GB]

On 04/10/2019 01:52, nd...@redhazel.co.uk wrote:

On 04/10/2019 00:26, Dave F via Talk-GB wrote:
I think you're missing the point. Most contributors believe postcodes 
on buildings or property nodes, add quality to the OSM's database, 
but object to the import of codepoint as it's just not accurate 
enough as stated in this, & numerous other threads.


This is incorrect. CPO/ONSPD postcodes _are_ accurate, up to date and 
include all postcodes in the UK except NI. 


Please note: "not accurate *enough*"

They are not complete (contain one and only one delivery point per 
postcode), 


No. The centre point is not associated with *any* delivery point. It is 
an arbitrary mean, calculated mathematically. it could, in theory, be 
located in the middle of a park.
Even postcodes unique to one property/business aren't accurate as their 
positions are misaligned by the effect of adjacent areas.


which is pity, but that's not a reason not to use the ones that are 
available, which is still _far_ more that what we have in the database.


Quantity does not equate to quality.



This may not be a perfect solution but the information CPO/ONSPD 
contains is still extremely useful for geocoding. Search for a 
postcode and you are _guaranteed_ to get an address in a close 
vicinity to a place you are looking for. 


No. With an import of the centroids points you're only guaranteed to be 
given the location of the node with the postcode.



How about not needing to start Google Maps when searching for a 
location on the go?


There's no point in importing to stand alone nodes as deliveries are 
destined for buildings. Adding to streets is also pointless for the 
same reason plus they can have multiple postcodes.


Addresses on nodes are commonly used in the UK OSM. Many mappers 
prefer them over placing addresses on buildings. There are also many 
cases (POIs) where nodes are objectively better than buildings. So, 
no, there right and wrong solution here.


Allow me to clarify. I should have maybe said 'properties' which can be 
represented by nodes instead of 'buildings'.
My objection, which I thought was clear, was to "standalone nodes" with 
just a postcode tag.





Besides, the main reason for importing these data is that we can get 
_all_ postcodes in the database.


Again quantity /= quality. If you can't manipulate data then it's 
useless. These standalone postcode nodes will relate to nothing.


This gives users confidence that when they search for a postcode they 
will reliably get a result they are looking for. This is not possible 
when merging postcodes with buildings simply because we still have 
only a small fraction of buildings in the database.


By the way, I'm not against merging addr:postcode with buildings, 
that's exactly what I was doing myself when adding postcodes manually. 
However, this is not a process that can be automated (lack of 
buildings, single OSM buildings having more than one address/postcode).


Then add buildings.

Based on my experience with mapping postcodes with CPO, I would 
recommend starting with an import and merge postcodes and buildings 
later.


Experience has shown that doesn't happen. I'm thinking US TIGER imports, 
but I'm sure there are other examples.


DaveF

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Import UK postcode data?

[Edward Bainton]

My usual disclaimer that I have no great OSM expertise. Hopefully I can
give a newbie's/recently-a-non-editor's perspective.

This:

> How about not needing to start Google Maps
> when searching for a location on the go?

would be reason enough to bring in postcodes in this way.

Even if they're not strictly 'on-the-ground', every rule needs exceptions
and the payoff from this exception would be _enormous_.

(And is anyway consistent with loc_name which is allowed even where the
name is nowhere written on signs on-the-ground?)

On Fri, 4 Oct 2019 at 11:20, David Woolley 
wrote:

> On 04/10/2019 00:26, Dave F via Talk-GB wrote:
> > There's no point in importing to stand alone nodes as deliveries are
> > destined for buildings. Adding to streets is also pointless for the same
> > reason plus they can have multiple postcodes.
>
> Deliveries are to specific points on buildings.  In principle they could
> also be to US style post boxes, on the property boundary.
>
> (Actually, the human entrance for a building might be in a different
> post code area from the mail delivery point, although I can't think of
> good examples.)
>
> ___
> Talk-GB mailing list
> Talk-GB@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-gb
>
___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb

Re: [Talk-GB] Import UK postcode data?

[Mateusz Konieczny]

4 Oct 2019, 01:26 by talk-gb@openstreetmap.org:
> There's no point in importing to stand alone nodes as deliveries are destined 
> for buildings. 
>
Importing accurate and complete address
nodes on a suitable licence is certainly
helpful.

Not sure is it applying also to incomplete
set, but based on my experience address
nodes are very helpful and allow to do less boring survey.

___
Talk-GB mailing list
Talk-GB@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-gb