On 28/09/2017 07:15, Stefano wrote:
We used this library to process the dump and then we add the results
in pgsql
https://www.entropywins.wtf/blog/2015/11/08/wikidata-wikibase-json-dump-reader/
https://github.com/osmItalia/wikidata-geo-match
What would the requirement of a wikidata2pgsql be?
Thanks for that.
To answer the question, It'd create a database in a format that's
designed to be queried that contains "just enough" information to
support whatever job it's needed for (and it'd be great if it also
supported dynamic column creation using a mechanism similar to
osm2pgsql's ".style" file).
Of the other "missing bits", I used "osmosis" as an example of "cutting
a database extract down to size" (other options are available). In the
OSM world that initial slice is often geographical, but osmosis can also
deal with data without explicit co-ordinates (ways and relations) based
on the geographical location of constituent nodes. The same would be
true (for me) of wikidata - I'd be only interested in actual physical
locations and the things that they link to (which may not have physical
locations and may just be concepts).
Finally "switch2osm" is a regularly-updated set of instructions that you
can follow from start to finish without needed external knowledge about
how to solve a problem. For example,
https://switch2osm.org/manually-building-a-tile-server-16-04-2-lts/ has
undergone numerous updates in the last year to deal with stylesheet
changes, which depend on a bleeding edge version of carto, which depends
on <BAD_WORD> node.js. At each stage in the process the idea was that
you'd always be able to get a working result, even if at one point that
meant the instructions explained how get a version of the stylesheet
from a few months ago because a newer version wouldn't work in
combination with everything else there.
"wikidata-wikibase-json-dump-reader" looks interesting - it looks (to
continue the analogy) to be somewhat equivalent to the Crosby PBF library.
"wikidata-geo-match" also looks interesting because
https://github.com/osmItalia/wikidata-geo-match/blob/master/scripts/2_process.sh
and the readme explain how to do the initial geographical selection.
It's not quite all there though
(https://github.com/osmItalia/wikidata-geo-match/search?utf8=%E2%9C%93&q=0_create_wikidata_table.sql&type=
suggests either the README is out of date or some bits are missing).
It'd certainly be a useful start for someone who cared about wikidata to
develop something mirroring the equivalent tools that OSM already has,
and I'm sure it does exactly what you need it to do, but it's not a
generic "let's create a database and allow you to do something with it".
Best Regards,
Andy
_______________________________________________
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
