Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-17 Thread Magnus Manske
Thanks, that looks great! On Tue, Mar 17, 2015 at 5:59 AM Stas Malyshev wrote: > Hi! > > I wrote a small translation tool from WDQ to SPARQL, which can be seen > here: > http://tools.wmflabs.org/wdq2sparql/ > > Currently it supports only one model of data and only subset of WDQ > syntax, but thi

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-16 Thread Stas Malyshev
Hi! I wrote a small translation tool from WDQ to SPARQL, which can be seen here: http://tools.wmflabs.org/wdq2sparql/ Currently it supports only one model of data and only subset of WDQ syntax, but this can be extended. I wrote it just as PoC to see how hard it would be (not too hard) and to see

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-12 Thread Nikolas Everett
On Wed, Mar 11, 2015 at 4:50 PM, Gabriel Wicke wrote: > My basic worries with exposing powerful query languages like SPARQL > publicly is that a) there is a large attack surface in the query processing > backend, and b) a client can request very expensive operations on the > server without perfor

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-12 Thread Stas Malyshev
Hi! > The choice for SPARQL was not made by me or by anyone who has a special > interest in pushing this particular formalism (in fact Nik and Stas can > confirm that I have been quite sceptical about the feasibility of using > BlazeGraph at first). It was the result of an open-minded discussion

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-11 Thread Gabriel Wicke
My basic worries with exposing powerful query languages like SPARQL publicly is that a) there is a large attack surface in the query processing backend, and b) a client can request very expensive operations on the server without performing much work itself. Timeouts can limit the damage, but if the

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-11 Thread Markus Krötzsch
On 11.03.2015 11:26, Daniel Kinzler wrote: Am 11.03.2015 um 10:43 schrieb Markus Krötzsch: I was referring to the investigations that have led to this spreadsheet: https://docs.google.com/a/wikimedia.org/spreadsheets/d/1MXikljoSUVP77w7JKf9EXN40OB-ZkMqT8Y5b2NYVKbU/edit#gid=0 That's the backend

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-11 Thread Daniel Kinzler
Am 11.03.2015 um 10:43 schrieb Markus Krötzsch: > I was referring to the investigations that have led to this spreadsheet: > > https://docs.google.com/a/wikimedia.org/spreadsheets/d/1MXikljoSUVP77w7JKf9EXN40OB-ZkMqT8Y5b2NYVKbU/edit#gid=0 That's the backend evaluation spreadsheet. I'm not arguing

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-11 Thread Daniel Kinzler
Am 11.03.2015 um 10:08 schrieb Markus Krötzsch: > What I don't see is how the use of a WDQ API on top of SPARQL would make the > overall setup any less vulnerable; it mainly introduces an additional > component > on top of SPARQL, and we can have a simpler SPARQL-based filter component > there >

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-11 Thread Markus Krötzsch
On 11.03.2015 05:59, Tom Morris wrote: On Tue, Mar 10, 2015 at 6:17 PM, Markus Krötzsch mailto:mar...@semantic-mediawiki.org>> wrote: TL;DR: No concrete issues with SPARQL were mentioned so far; OTOH many *simple* SPARQL queries are not possible in WDQ; there is still time to restric

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-11 Thread Markus Krötzsch
On 11.03.2015 00:44, Magnus Manske wrote: To be fair, the discussion is not "what will we do till the end of time", rather "what do we start with". Knowing neither SPARQL nor the data storage engine terribly well, it would not be helpful if the service can be DOSed by innocent-looking queries, i

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-11 Thread Magnus Manske
On Wed, Mar 11, 2015 at 4:52 AM Tom Morris wrote: > How long has WDQ been in service? > > Before September 2013. So, 1.5-2 years. ___ Wikidata-tech mailing list Wikidata-tech@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-tech

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-10 Thread Tom Morris
On Tue, Mar 10, 2015 at 6:17 PM, Markus Krötzsch < mar...@semantic-mediawiki.org> wrote: > TL;DR: No concrete issues with SPARQL were mentioned so far; OTOH many > *simple* SPARQL queries are not possible in WDQ; there is still time to > restrict ourselves -- let's give SPARQL a chance before goin

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-10 Thread Tom Morris
How long has WDQ been in service? What proportion of the total aggregate lifetime Wikidata apps, presuming it survives, do the current, as of Mar 2015, Wikidata apps represent? Should the question of premature optimization (or optimisation) be considered? Tom p.s. Since your opinion doesn't rep

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-10 Thread Magnus Manske
To be fair, the discussion is not "what will we do till the end of time", rather "what do we start with". Knowing neither SPARQL nor the data storage engine terribly well, it would not be helpful if the service can be DOSed by innocent-looking queries, intentional or not. Exposing only a subset of

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-10 Thread Markus Krötzsch
TL;DR: No concrete issues with SPARQL were mentioned so far; OTOH many *simple* SPARQL queries are not possible in WDQ; there is still time to restrict ourselves -- let's give SPARQL a chance before going back. Hi Daniel, This discussion is way too abstract. I am missing hard facts about the

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-10 Thread Daniel Kinzler
Am 10.03.2015 um 21:09 schrieb Stas Malyshev: > People would ask us for full SPARQL as soon as they'd know we're > running SPARQL db. Sure. And I'D tell them "you can use SPARQL on labs, but beware that it may change or go away". -- Daniel Kinzler Senior Software Developer Wikimedia Deutschlan

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-10 Thread Stas Malyshev
Hi! > So, my proposal is to expose a WDQ-like service as our primary query > interface. > This follows the general principle having narrow interfaces to make it easy to > swap out the implementation. WDQ query language is somewhat limited as I understand. We can of write WDQ->SPARQL translator,

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-10 Thread Daniel Kinzler
Am 10.03.2015 um 18:22 schrieb Thomas Tanon: > I support Magnus point of view. WDQ is a very good proof of concept but is, > I think, to limited to be the primary language of the Wikidata query > system. It can be extended. What I want is a limited domain specific language tailored to our primary

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-10 Thread Thomas Tanon
I support Magnus point of view. WDQ is a very good proof of concept but is, I think, to limited to be the primary language of the Wikidata query system. A possible solution is maybe to support two query languages "as primary": 1 WDQ, at first, in order to have something working quickly 2 A safe s

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-10 Thread Magnus Manske
Some thoughts: * Either way, there will be a WDQ-like wrapper around SPARQL, maybe as the official interface, maybe only at the current WDQ URL (and I'll have to read up on SPARQL to write that, so if someone else writes it for me, all the better!) * WDQ syntax is very limited (no references, no va

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-10 Thread Daniel Kinzler
Am 10.03.2015 um 16:47 schrieb Markus Krötzsch: > Hi Daniel, > > I can understand your thoughts to some extent, but they seem to apply to any > potential solution. Committing to a primary query interface will always be, > well, a committment. Because of this, I think the big advantage of SPARQL is

Re: [Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-10 Thread Markus Krötzsch
Hi Daniel, I can understand your thoughts to some extent, but they seem to apply to any potential solution. Committing to a primary query interface will always be, well, a committment. Because of this, I think the big advantage of SPARQL is exactly that it is a technology standard that is not

[Wikidata-tech] Thoughts on (not) exposing a SPARQL endpoint

2015-03-10 Thread Daniel Kinzler
Hi all! After the initial enthusiasm, I have grown increasingly wary of the prospect of exposing a SPARQL endpoint as Wikidata's canonical query interface. I decided to share my (personal and unfinished) thoughts about this on this list, as food for thought and a basis for discussion. Basically,