Re: [OSM-talk] new Wikidata+OSM data in one RDF database

[Yuri Astrakhan]

Thanks to quick fix by Sarah, the OSM updater is now working, and will
catch up shortly.  I am still looking for a permanent home for this
service, as I am pretty sure it would be highly useful especially for tag
analysis and data validation.

mmd, thanks!! I was asking earlier and was told that there is no way to do
complex comparisons between tags in different objects. I really hope we
could teach SPARQL to do all the amazing geometry work that Overpass can
do, but I suspect there are significant number of queries that are easier
to express in SPARQL.  If only we can join the two :)

On Thu, May 25, 2017 at 3:06 AM mmd  wrote:

> Hi,
>
> Am 25.05.2017 um 08:50 schrieb Yuri Astrakhan:
> > The service is back up, this time with all the objects that have tags.
> > Also, I added the "has" properties on a relation - indicating all
> > objects contained within the relation.  So now you can ask for a
> > relation, that contains a way, and both the relation and the way have
> > the same wikidata ID (something you cannot get from overpass):
> >
> > http://tinyurl.com/k4vjkje
> >
>
> Sure you can do this with overpass: http://overpass-turbo.eu/s/phj
>
> The example returns relation 416351 along with all ways having the same
> wikidata id. I commented out the part to check all relations in the
> current bounding box, but I guess you'll get the idea.
>
> best,
> mmd
>
>
>
>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] new Wikidata+OSM data in one RDF database

[mmd]

Hi,

Am 25.05.2017 um 08:50 schrieb Yuri Astrakhan:
> The service is back up, this time with all the objects that have tags. 
> Also, I added the "has" properties on a relation - indicating all
> objects contained within the relation.  So now you can ask for a
> relation, that contains a way, and both the relation and the way have
> the same wikidata ID (something you cannot get from overpass):
> 
> http://tinyurl.com/k4vjkje
> 

Sure you can do this with overpass: http://overpass-turbo.eu/s/phj

The example returns relation 416351 along with all ways having the same
wikidata id. I commented out the part to check all relations in the
current bounding box, but I guess you'll get the idea.

best,
mmd





___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] new Wikidata+OSM data in one RDF database

[h.exterkate]

Y


Verzonden vanaf mijn Samsung Galaxy-smartphone.
 Oorspronkelijk bericht Van: Sarah Hoffmann <lon...@denofr.de> 
Datum: 25-05-17  09:51  (GMT+01:00) Aan: Yuri Astrakhan 
<yuriastrak...@gmail.com> Cc: Kolossos <t...@alder-digital.de>, 
talk@openstreetmap.org Onderwerp: Re: [OSM-talk] new Wikidata+OSM data in one 
RDF database 
That is quite obviously a bug. For progress on fixing it see
https://github.com/osmcode/pyosmium/issues/38. Please take
into account that the maintainers do sleep from time to time
which might explain why they don't answer immediately. ;)

Kind regards

Sarah

On Thu, May 25, 2017 at 06:54:27AM +, Yuri Astrakhan wrote:
> P.S. I am trying to get OSM updater to work, so that OSM data is always up
> to date, but pyosmium is giving me some trouble. Please email if you know
> the answer to
> https://stackoverflow.com/questions/44170360/callbacks-not-called-in-pyosmiums-diff-downloader
> 
> On Wed, May 24, 2017 at 11:50 PM Yuri Astrakhan <yuriastrak...@gmail.com>
> wrote:
> 
> > The service is back up, this time with all the objects that have tags.
> > Also, I added the "has" properties on a relation - indicating all objects
> > contained within the relation.  So now you can ask for a relation, that
> > contains a way, and both the relation and the way have the same wikidata ID
> > (something you cannot get from overpass):
> >
> > http://tinyurl.com/k4vjkje
> >
> > "has" could be in one of three forms:
> > ?osmObject1  osmm:has  ?osmObject2   # obj1 contains obj2, no label is set
> > ?osmObject1  osmm:has:inner  ?osmObject2  # can also be outer,
> > center_admin, etc.
> > ?osmObject1  osmm:has:_  ?osmObject2  # the label is not simple ascii,
> > and should be fixed
> >
> >
> > On Mon, May 22, 2017 at 2:04 AM Janko Mihelić <jan...@gmail.com> wrote:
> >
> >> Wow, I think this is a great milestone. Thanks!
> >>
> >> Now if only we can get a mixture of Wikidata's SPARQL and Overpass QL. A
> >> kind of a hybrid language between the two? Because Wikidata will probably
> >> never have the Overpass "in" or "around", which narrows the data down to a
> >> single country or county, or to a radius around something. I find that very
> >> useful.
> >>
> >> What if you connected to the Overpass API, ran the Overpass query, and
> >> then filtered the Wikidata data by the results of Overpass? Does that even
> >> make sense? For example: Overpass gives me all elements with a wikidata tag
> >> in a county, and then SPARQL can filter down the data to find all humans
> >> within that data. I think that's possible.
> >>
> >> Anyway, thanks for your service (although I think it's down right now).
> >>
> >> Janko
> >>
> >> ___
> >> talk mailing list
> >> talk@openstreetmap.org
> >> https://lists.openstreetmap.org/listinfo/talk
> >>
> >

> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] new Wikidata+OSM data in one RDF database

[Sarah Hoffmann]

That is quite obviously a bug. For progress on fixing it see
https://github.com/osmcode/pyosmium/issues/38. Please take
into account that the maintainers do sleep from time to time
which might explain why they don't answer immediately. ;)

Kind regards

Sarah

On Thu, May 25, 2017 at 06:54:27AM +, Yuri Astrakhan wrote:
> P.S. I am trying to get OSM updater to work, so that OSM data is always up
> to date, but pyosmium is giving me some trouble. Please email if you know
> the answer to
> https://stackoverflow.com/questions/44170360/callbacks-not-called-in-pyosmiums-diff-downloader
> 
> On Wed, May 24, 2017 at 11:50 PM Yuri Astrakhan 
> wrote:
> 
> > The service is back up, this time with all the objects that have tags.
> > Also, I added the "has" properties on a relation - indicating all objects
> > contained within the relation.  So now you can ask for a relation, that
> > contains a way, and both the relation and the way have the same wikidata ID
> > (something you cannot get from overpass):
> >
> > http://tinyurl.com/k4vjkje
> >
> > "has" could be in one of three forms:
> > ?osmObject1  osmm:has  ?osmObject2   # obj1 contains obj2, no label is set
> > ?osmObject1  osmm:has:inner  ?osmObject2  # can also be outer,
> > center_admin, etc.
> > ?osmObject1  osmm:has:_  ?osmObject2  # the label is not simple ascii,
> > and should be fixed
> >
> >
> > On Mon, May 22, 2017 at 2:04 AM Janko Mihelić  wrote:
> >
> >> Wow, I think this is a great milestone. Thanks!
> >>
> >> Now if only we can get a mixture of Wikidata's SPARQL and Overpass QL. A
> >> kind of a hybrid language between the two? Because Wikidata will probably
> >> never have the Overpass "in" or "around", which narrows the data down to a
> >> single country or county, or to a radius around something. I find that very
> >> useful.
> >>
> >> What if you connected to the Overpass API, ran the Overpass query, and
> >> then filtered the Wikidata data by the results of Overpass? Does that even
> >> make sense? For example: Overpass gives me all elements with a wikidata tag
> >> in a county, and then SPARQL can filter down the data to find all humans
> >> within that data. I think that's possible.
> >>
> >> Anyway, thanks for your service (although I think it's down right now).
> >>
> >> Janko
> >>
> >> ___
> >> talk mailing list
> >> talk@openstreetmap.org
> >> https://lists.openstreetmap.org/listinfo/talk
> >>
> >

> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] new Wikidata+OSM data in one RDF database

[Yuri Astrakhan]

P.S. I am trying to get OSM updater to work, so that OSM data is always up
to date, but pyosmium is giving me some trouble. Please email if you know
the answer to
https://stackoverflow.com/questions/44170360/callbacks-not-called-in-pyosmiums-diff-downloader

On Wed, May 24, 2017 at 11:50 PM Yuri Astrakhan 
wrote:

> The service is back up, this time with all the objects that have tags.
> Also, I added the "has" properties on a relation - indicating all objects
> contained within the relation.  So now you can ask for a relation, that
> contains a way, and both the relation and the way have the same wikidata ID
> (something you cannot get from overpass):
>
> http://tinyurl.com/k4vjkje
>
> "has" could be in one of three forms:
> ?osmObject1  osmm:has  ?osmObject2   # obj1 contains obj2, no label is set
> ?osmObject1  osmm:has:inner  ?osmObject2  # can also be outer,
> center_admin, etc.
> ?osmObject1  osmm:has:_  ?osmObject2  # the label is not simple ascii,
> and should be fixed
>
>
> On Mon, May 22, 2017 at 2:04 AM Janko Mihelić  wrote:
>
>> Wow, I think this is a great milestone. Thanks!
>>
>> Now if only we can get a mixture of Wikidata's SPARQL and Overpass QL. A
>> kind of a hybrid language between the two? Because Wikidata will probably
>> never have the Overpass "in" or "around", which narrows the data down to a
>> single country or county, or to a radius around something. I find that very
>> useful.
>>
>> What if you connected to the Overpass API, ran the Overpass query, and
>> then filtered the Wikidata data by the results of Overpass? Does that even
>> make sense? For example: Overpass gives me all elements with a wikidata tag
>> in a county, and then SPARQL can filter down the data to find all humans
>> within that data. I think that's possible.
>>
>> Anyway, thanks for your service (although I think it's down right now).
>>
>> Janko
>>
>> ___
>> talk mailing list
>> talk@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk
>>
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] new Wikidata+OSM data in one RDF database

[Yuri Astrakhan]

The service is back up, this time with all the objects that have tags.
Also, I added the "has" properties on a relation - indicating all objects
contained within the relation.  So now you can ask for a relation, that
contains a way, and both the relation and the way have the same wikidata ID
(something you cannot get from overpass):

http://tinyurl.com/k4vjkje

"has" could be in one of three forms:
?osmObject1  osmm:has  ?osmObject2   # obj1 contains obj2, no label is set
?osmObject1  osmm:has:inner  ?osmObject2  # can also be outer,
center_admin, etc.
?osmObject1  osmm:has:_  ?osmObject2  # the label is not simple ascii, and
should be fixed


On Mon, May 22, 2017 at 2:04 AM Janko Mihelić  wrote:

> Wow, I think this is a great milestone. Thanks!
>
> Now if only we can get a mixture of Wikidata's SPARQL and Overpass QL. A
> kind of a hybrid language between the two? Because Wikidata will probably
> never have the Overpass "in" or "around", which narrows the data down to a
> single country or county, or to a radius around something. I find that very
> useful.
>
> What if you connected to the Overpass API, ran the Overpass query, and
> then filtered the Wikidata data by the results of Overpass? Does that even
> make sense? For example: Overpass gives me all elements with a wikidata tag
> in a county, and then SPARQL can filter down the data to find all humans
> within that data. I think that's possible.
>
> Anyway, thanks for your service (although I think it's down right now).
>
> Janko
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] new Wikidata+OSM data in one RDF database

[Janko Mihelić]

Wow, I think this is a great milestone. Thanks!

Now if only we can get a mixture of Wikidata's SPARQL and Overpass QL. A
kind of a hybrid language between the two? Because Wikidata will probably
never have the Overpass "in" or "around", which narrows the data down to a
single country or county, or to a radius around something. I find that very
useful.

What if you connected to the Overpass API, ran the Overpass query, and then
filtered the Wikidata data by the results of Overpass? Does that even make
sense? For example: Overpass gives me all elements with a wikidata tag in a
county, and then SPARQL can filter down the data to find all humans within
that data. I think that's possible.

Anyway, thanks for your service (although I think it's down right now).

Janko
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] new Wikidata+OSM data in one RDF database

[Oleksiy Muzalyev]

Dear Rob and Yuri,

I have read your messages with great interest. I hope this work will 
lead eventually to a Wikipedia articles multi-language layer on the OSM 
map, to the navigation apps which are capable not only tell where to 
turn but also about places of interest and historical events one drives 
or walks by. Often it is not necessary to travel thousands of kilometers 
to see something interesting, a lot of amazing places are close to us.


I wrote a web-application which allows to locate Wikipedia articles with 
geographical coordinates in a selected language (or wikidata tags): 
http://ausleuchtung.ch/geo_wiki/ . It is based on the MediaWiki Api and 
the Overpass Api. Certainly, having a single united database would allow 
queries which are hard to imagine yet.


In my opinion the multi-language capabilities are essential for such 
OSM tools. A lot of absolutely interesting articles with 
geographical coordinates exist only in one language [1]. Fortunately 
Wikipedia community subscribes to the idea that the true international 
language is translation, and there are Wikipedia tools to translate an 
article in any supported language relatively easily.


Please, keep me informed of further development of your projects.

[1] https://fr.wikipedia.org/wiki/Fort_du_Risoux
https://fr.wikipedia.org/wiki/Tuilerie_romaine_des_Bois_de_Chancy
etc.

Thank you and best regards,
Oleksiy

On 14.05.2017 20:10, Rob H Warren wrote:

Yuri,

There is a plan afoot to do something similar with the geometries from 
www.openhistoricalmap.org; let's keep in touch, I would really like to enable 
linkages across datasets.

Keep up the good work! -rhw



On May 13, 2017, at 8:00 AM, talk-requ...@openstreetmap.org wrote:


Message: 1
Date: Fri, 12 May 2017 16:03:52 +
From: Yuri Astrakhan 
To: OpenStreetMap talk mailing list 
Subject: [OSM-talk] new Wikidata+OSM data in one RDF database
Message-ID:

Re: [OSM-talk] new Wikidata+OSM data in one RDF database

[Kolossos]

Nice work Yuri,
I hope the next version will have geometries in GeoSPARQL and an output
directly on a map, but one step after another.

I believe for the moment it's not so important to have all the OSM
objects without Wikidata link inside the database. To handle this amount
of data would be much harder.

Very cool project.

Best regards
Tim alias Kolossos



Am 12.05.2017 um 18:03 schrieb Yuri Astrakhan:
> TLDR: A SPARQL (rdf) database with both OSM and Wikidata data is up for
> testing.  Allows massive cross-referenced queries between two datasets.
> The service is a test, and needs a permanent home to stay alive.
> 
> Overpass Turbo is awesome, but sadly it does not have data from
> Wikidata, nor does it support some SQL-like conditions. I have setup a
> temporary RDF database that has both OSM & Wikidata. You can use SPARQL
> queries to find:
> 
> * All OSM objects with wikidata tag that references a Wikipedia
> disambiguation page. Get the name of the page in first available
> language ru, fr, de, en.http://tinyurl.com/mzlfb26
> 
> * OSM relations with wikidata tag pointing to a person (also tries
> multiple language fallbacks).  http://tinyurl.com/m6fh3wx
> 
> * OSM relations with duplicate Wikidata IDs http://tinyurl.com/mvhhogx
> 
> 
> == OSM data structure ==
> osmnode, osmway, osmrel - OSM object prefix, e.g.  osmnode:1234
> osmt - tag, e.g.  osmt:name:en  (only has tags with latin chars, -, _,
> :, digits
> osmm - meta data about the object -- type, isClosed, version.
> 
> I try to preserve OSM data without much changes. Every tag's value is
> stored as a string, except for wikidata and wikipedia tags which are
> converted to a URL, the same format as stored in Wikidata.
> 
> osmway:29453885
>   osmt:name "Samina";
>   osmt:waterway "river";
>   osmt:wikidata wd:Q156065;
>   osmt:wikipedia ;
>   osmm:type "w";    could be "r", "w", and "n"
>   osmm:isClosed false;     this meta property is only for OSM ways
>   osmm:version 24.
> 
> Wikidata data structure is identical to https://query.wikidata.org (see
> help)
> 
> 
> == Current limitations ==
> * Only includes OSM objects with either "wikidata" or "wikipedia" tags
> * The OSM data only contains tags with only Latin letters, digits and
> symbols - : _  
> * OSM geometry info is not imported, e.g. no center point or bounding
> box, except for osmm:isClosed (true/false) property for ways. 
> * Does not include OSM object inheritance data - e.g. cannot query for
> "find a node that is part of a way which is part of a relation that has
> wikidata tag that ..."
> * Wikidata is updated every second, but OSM does not yet update at all,
> imported from a full db dump as of a few days ago.
> 
> 
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
> 



___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] new Wikidata+OSM data in one RDF database

[Eugene Alvin Villar]

So what is the license of this combined database and is this combined
database a whole database or a collective database? Note that Wikidata is
under CC0 while OSM is under ODbL.

On Sat, May 13, 2017 at 12:03 AM, Yuri Astrakhan 
wrote:

> TLDR: A SPARQL (rdf) database with both OSM and Wikidata data is up for
> testing.  Allows massive cross-referenced queries between two datasets. The
> service is a test, and needs a permanent home to stay alive.
>
> Overpass Turbo is awesome, but sadly it does not have data from Wikidata,
> nor does it support some SQL-like conditions. I have setup a temporary RDF
> database that has both OSM & Wikidata. You can use SPARQL queries to find:
>
> * All OSM objects with wikidata tag that references a Wikipedia
> disambiguation page. Get the name of the page in first available language
> ru, fr, de, en.http://tinyurl.com/mzlfb26
>
> * OSM relations with wikidata tag pointing to a person (also tries
> multiple language fallbacks).  http://tinyurl.com/m6fh3wx
>
> * OSM relations with duplicate Wikidata IDs http://tinyurl.com/mvhhogx
>
>
> == OSM data structure ==
> osmnode, osmway, osmrel - OSM object prefix, e.g.  osmnode:1234
> osmt - tag, e.g.  osmt:name:en  (only has tags with latin chars, -, _, :,
> digits
> osmm - meta data about the object -- type, isClosed, version.
>
> I try to preserve OSM data without much changes. Every tag's value is
> stored as a string, except for wikidata and wikipedia tags which are
> converted to a URL, the same format as stored in Wikidata.
>
> osmway:29453885
>   osmt:name "Samina";
>   osmt:waterway "river";
>   osmt:wikidata wd:Q156065;
>   osmt:wikipedia ;
>   osmm:type "w";    could be "r", "w", and "n"
>   osmm:isClosed false;     this meta property is only for OSM ways
>   osmm:version 24.
>
> Wikidata data structure is identical to https://query.wikidata.org (see
> help)
>
>
> == Current limitations ==
> * Only includes OSM objects with either "wikidata" or "wikipedia" tags
> * The OSM data only contains tags with only Latin letters, digits and
> symbols - : _
> * OSM geometry info is not imported, e.g. no center point or bounding box,
> except for osmm:isClosed (true/false) property for ways.
> * Does not include OSM object inheritance data - e.g. cannot query for
> "find a node that is part of a way which is part of a relation that has
> wikidata tag that ..."
> * Wikidata is updated every second, but OSM does not yet update at all,
> imported from a full db dump as of a few days ago.
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk