Fixing stuff in OSM purely from imagery may not be good.
A local mapper who sees something may add it before any satellite imagery has
it.
If you then 'fix' this back to the satellite imagery you will have committed an
error,
and that error may dissuade our most important resource from ever making any
further changes- the local mapper.
Be very careful!
On 10/3/20 1:11 pm, Michal Migurski wrote:
Hi Joseph, thanks for your questions!
We are unsure yet about the public release schedule we want to commit to,
because a lot of it depends on community feedback which we’re getting now. I
hope we will do this again!
We do correct OSM upstream for the errors that we find. When our human review
process catches a map error, we do two things:
1) Hold it back from release to our display maps
2) Fix the error upstream in OSM.org
Ideally, the fix subsequently passes our process on the next round.
-mike.
On Mar 9, 2020, at 5:47 PM, Joseph Eisenberg <joseph.eisenb...@gmail.com> wrote:
Interesting! That sounds like a huge amount of work, if you are
validating every changeset in the globe, though so far there is no
schedule to release this frequently.
Thank you for following the Open Database license by releasing this to
the public.
Is facebook planning to create new planet files like this on a
frequent basis, for internal use? If so, those should be released to
the public on a regular schedule, if I understand the ODbL correctly.
Can I confirm that facebook is editing Openstreetmap data to fix the
errors which are found? Or is this only happening in some cases?
The diary post gave links that explain how this was made (both from
September 2019):
- https://engineering.fb.com/ml-applications/mars/ "MaRS: How
Facebook keeps maps current and accurate"
"We calculate the absolute diff between the two OSM versions (from
option 1 above).
We then split this diff into a set of LoChas that can be individually
applied (from option 2)."
...
"At a high level, a map change is vetted by a reviewer,"
-
https://2019.stateofthemap.us/program/sun/keepin-it-fresh-and-good-continuous-ingestion-of-osm-data-at-facebook.html
"“Keepin’ it fresh (and good)!” - Continuous Ingestion of OSM Data at
Facebook":
This is a video presentation, so I haven't seen it all, but the
abstract says: "we have created an automated ingestion and integrity
framework for OSM data that allows us to selectively update parts of
the map instead of doing a full snapshot change all at once.
"Decomposing a large set of changes in this way gives us the
flexibility to rapidly ingest our own additions to the map, focus on
geographical areas of importance to downstream products, and allows us
to quickly apply hotfixes whenever egregious problems do arise.
"With millions of tiny changes happening every week, we have created a
system that is built on per-feature approval and preprocessing, that
allows us to ingest changes at scale, while creating rules to
automatically process logical changesets and enforce integrity
constraints (e.g. anti-vandalism, anti-profanity etc.).
"Due to the contextual nature of some of the changes in OpenStreetMap,
the system combines Human Approval, necessary for highly visible
features such as names of large administrative areas, with Automated
AI/ML-based approval: for example, using computer vision techniques to
reconcile newly created features against satellite imagery ground
truth, or applying NLP techniques to determine whether other
user-visible string changes are sensible and valid. These components
are combined to create a continuous ingest-validate-deploy cycle for
OSM map data."
Lot's of buzz words there! But it sounds like it is a combination of
computer algorithms and human checking for vandalism and errors.
- Joseph Eisenberg
On 3/10/20, Michal Migurski <m...@teczno.com> wrote:
Hi everyone,
I’m writing to let you know about a new OpenStreetMap project Facebook just
released. It’s called Daylight Map Distribution. Daylight is a complete,
downloadable preview of OpenStreetMap data we plan to start using in a
number of our public maps:
https://www.openstreetmap.org/user/migurski/diary/392416
Facebook uses maps to let our users find friends, businesses, groups and
more. OpenStreetMap (OSM) has a substantial global footprint of map data
built and maintained by a dedicated community of global mappers and it’s a
natural choice for us. Every day, OSM receives millions of contributions
from the community. Some of these contributions may have intentional and
unintentional edits that are incompatible with our needs. Our mapping teams
work to scrub these contributions for consistency and quality.
What’s Included in the Daylight Map Distribution:
• A PBF planet file composed of 100% OSM data, released under the terms
of
the Open Database License.
• Only those edits which have been validated to contain no malicious
vandalism or unintentional errors so we can show them in our display maps
This is just an initial first release, and we’re looking for feedback from
the community to decide what would be useful to release in the future and
how frequently. I’d be interested to hear any response you might have about
it!
-mike.
----------------------------------------------------------------
michal migurski- contact info and pgp key:
sf/ca http://mike.teczno.com/contact.html
_______________________________________________
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
