Re: [Toolserver-l] Wikidata tables

2013-04-22 Thread Daniel Schwen
> FWIW, I have implemented a query-able stand-alone web server that keeps all > of the wikidata property-item-links in memory. This uses the wikidata dumps That does not sound too terribly scalable. I did the same thing (custom webserver, data kept in memory) for the map labels of my WikiMiniAtlas

Re: [Toolserver-l] Wikidata tables

2013-04-19 Thread Patricia Pintilie
Waterfall the Anamorphic Development. On Apr 19, 2013 4:14 AM, "Patricia Pintilie" wrote: > Ok so how about we recocnize what the overal goal is first . Then > establish the point that its trying to convay. Only then can we meet in the > middle and set a plan in motion. I can only assist when a p

Re: [Toolserver-l] Wikidata tables

2013-04-19 Thread Platonides
Sorry, but your mail doesn't make much sense. El 19/04/13 11:14, Patricia Pintilie escribió: > Ok so how about we recocnize what the overal goal is first . Then > establish the point that its trying to convay. Only then can we meet in > the middle and set a plan in motion. I can only assist when a

Re: [Toolserver-l] Wikidata tables

2013-04-19 Thread Patricia Pintilie
Ok so how about we recocnize what the overal goal is first . Then establish the point that its trying to convay. Only then can we meet in the middle and set a plan in motion. I can only assist when a plan of action is clear with a definite plan without it im lost on where to begin. It seems as if i

Re: [Toolserver-l] Wikidata tables

2013-04-19 Thread Kolossos
Sounds fine, but will it be possible to join the data with data from other tables and other projects? This joins are the base for a lot of tools on toolserver and I#m not sure how good joins on application level will work. BTW: With the project Templatetiger we already handle tons of informat

Re: [Toolserver-l] Wikidata tables

2013-04-19 Thread Kolossos
Hello, Postrgesql starts now to support also JSON, so we should try to find a way to bring Wikidata available for us and I would prefer to use furthermore SQL. One way could be minutely diff-files, that's the way OpenStreetMap use. Alternatively we could use API for each updated article. Ever

Re: [Toolserver-l] Wikidata tables

2013-04-19 Thread Magnus Manske
FWIW, I have implemented a query-able stand-alone web server that keeps all of the wikidata property-item-links in memory. This uses the wikidata dumps which appear to be rather frequent. I'll try do deploy a test version on wikilabs (once I figure out how all that works); it seems to be more favou

Re: [Toolserver-l] Wikidata tables

2013-04-19 Thread Platonides
On 19/04/13 01:19, DaB. wrote: > as you may know there is a rev_text_id-field in the revision-table. This > field > points to the text-table where the actual text is – or should be. Because the > WMF doesn’t store the text here, but only a pointer > ("DB://cluster25/11458305" > for example). I

Re: [Toolserver-l] Wikidata tables

2013-04-18 Thread DaB.
Hello, At Friday 19 April 2013 01:03:25 DaB. wrote: > > would be no way to separate Wikidata from the rest > > I don't understand why separating plaintext storage between different > projects would be an issue. Is it all lumped into one storage > "namespace"? > I'm sure nobody at Wikimedia would b

Re: [Toolserver-l] Wikidata tables

2013-04-18 Thread Daniel Schwen
> if these JSON-data is stored where the normal wiki-text is, it is imposable To my understanding it is. > for us to replicate it: Because we have no access to these wmf-servers, there IMO that was a questionable design decision. JSON plaintext storage in SQL is shoehorning a do-it-yourself object

Re: [Toolserver-l] Wikidata tables

2013-04-18 Thread DaB.
Hello, At Thursday 18 April 2013 22:00:54 DaB. wrote: > but the JSON data you are talking about does not seem > copyrightable and much lower in volume. if these JSON-data is stored where the normal wiki-text is, it is imposable for us to replicate it: Because we have no access to these wmf-serve

Re: [Toolserver-l] Wikidata tables

2013-04-18 Thread Daniel Schwen
I can (barely) understand why wikitext is not available on the toolserver, but the JSON data you are talking about does not seem copyrightable and much lower in volume. The usefulness of the wikidata mirror on the toolserver is rather low without the actual wikiDATA. Daniel On Thu, Apr 18, 2013 at

Re: [Toolserver-l] Wikidata tables

2013-04-18 Thread Byrial Jensen
Den 18-04-2013 11:21, Lydia Pintscher skrev: On Thu, Apr 18, 2013 at 9:52 AM, Magnus Manske wrote: Just wondering what the status of exposing all wikidata tables on the toolserver is. Currently, there are a few wb_* tables with item labels, descriptions, aliases, and language links. But the t

Re: [Toolserver-l] Wikidata tables

2013-04-18 Thread Patricia Pintilie
Imagine Magnus all the files are being looked over to determine human, machine, and non used accounts. Each needs to be looked over then proper deletion will clean out making more room. Problematic yes but will help if this is taken care of the right way now. On Apr 18, 2013 5:45 AM, "Magnus Manske

Re: [Toolserver-l] Wikidata tables

2013-04-18 Thread Magnus Manske
Huh. That could be ... problematic in the future. Thanks, Magnus On Thu, Apr 18, 2013 at 10:21 AM, Lydia Pintscher < lydia.pintsc...@wikimedia.de> wrote: > On Thu, Apr 18, 2013 at 9:52 AM, Magnus Manske > wrote: > > Just wondering what the status of exposing all wikidata tables on the > > tool

Re: [Toolserver-l] Wikidata tables

2013-04-18 Thread Lydia Pintscher
On Thu, Apr 18, 2013 at 9:52 AM, Magnus Manske wrote: > Just wondering what the status of exposing all wikidata tables on the > toolserver is. > > Currently, there are a few wb_* tables with item labels, descriptions, > aliases, and language links. > > But the tables (whatever they are called) con

[Toolserver-l] Wikidata tables

2013-04-18 Thread Magnus Manske
Just wondering what the status of exposing all wikidata tables on the toolserver is. Currently, there are a few wb_* tables with item labels, descriptions, aliases, and language links. But the tables (whatever they are called) containing item-to-item connections appear to be missing. Maybe becaus