Re: Again on endpoint server limits [WAS Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.]

2013-06-03 Thread Andy Seaborne
There's a practical tradeoff of streaming and whole result processing in the server. Streaming can give lower latency to first result for the client which can be a better user experience. HTTP status code go in the header so to insert the perfect answer, the server needs to see the end of the

Re: Again on endpoint server limits [WAS Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.]

2013-05-31 Thread Kingsley Idehen
On 5/31/13 11:17 AM, Mark Baker wrote: On Thu, May 30, 2013 at 10:33 AM, Kingsley Idehen wrote: On 5/30/13 9:13 AM, Andrea Splendiani wrote: Hi, let me get back to this thread for two reasons. 1) I was wondering whether the report on DBPedia queries cited below was already published. 2) I hav

Re: Again on endpoint server limits [WAS Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.]

2013-05-31 Thread Mark Baker
On Thu, May 30, 2013 at 10:33 AM, Kingsley Idehen wrote: > On 5/30/13 9:13 AM, Andrea Splendiani wrote: >> >> Hi, >> >> let me get back to this thread for two reasons. >> 1) I was wondering whether the report on DBPedia queries cited below was >> already published. >> 2) I have recently tried to u

Re: Again on endpoint server limits [WAS Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.]

2013-05-30 Thread Kingsley Idehen
On 5/30/13 10:42 AM, Andrea Splendiani wrote: Hi, good. A different http header would be good. The problem is that, in a typical application (or at least some of them) you don't really know which server is there, so specific headers (non http) may not be known. More in general, I think the wi

Re: Again on endpoint server limits [WAS Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.]

2013-05-30 Thread Andrea Splendiani
Hi, good. A different http header would be good. The problem is that, in a typical application (or at least some of them) you don't really know which server is there, so specific headers (non http) may not be known. More in general, I think the wider public won't be so precise: either you get

Re: Again on endpoint server limits [WAS Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.]

2013-05-30 Thread Kingsley Idehen
On 5/30/13 9:13 AM, Andrea Splendiani wrote: Hi, let me get back to this thread for two reasons. 1) I was wondering whether the report on DBPedia queries cited below was already published. 2) I have recently tried to use DBPedia for some simple computation and I have a problem. Basically a que

Again on endpoint server limits [WAS Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.]

2013-05-30 Thread Andrea Splendiani
Hi, let me get back to this thread for two reasons. 1) I was wondering whether the report on DBPedia queries cited below was already published. 2) I have recently tried to use DBPedia for some simple computation and I have a problem. Basically a query for all cities whose population is larger th

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-30 Thread Jerven Bolleman
Hi Everyone, On Fri, Apr 19, 2013 at 1:06 AM, Andrea Splendiani < andrea.splendi...@iscb.org> wrote: > Il giorno 18/apr/2013, alle ore 16:04, Kingsley Idehen < > kide...@openlinksw.com> ha scritto: > > > On 4/18/13 9:23 AM, Andrea Splendiani wrote: > >> Hi, > >> > >> I think that some caching wit

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-19 Thread Mark Baker
On Fri, Apr 19, 2013 at 9:25 AM, Rob Warren wrote: >> Hi Rob, >> >> There is a fundamental problem with HTTP status codes. >> Lets say a user submits a complex but small sparql request. >> >> My server sees the syntax is good and starts to reply in good faith. >> This means the server starts the h

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-19 Thread Rob Warren
On 19-Apr-13, at 8:04 AM, Leigh Dodds wrote: Hi, On Fri, Apr 19, 2013 at 11:55 AM, Kingsley Idehen wrote: ... If you have OFFSET and LIMIT in use, you can reflect the new state of affairs when the next GET is performed i.e, lets say you have OFFSET 20 and LIMIT 20, the URL with OFFSET 40 is

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-19 Thread Kingsley Idehen
On 4/19/13 9:25 AM, Rob Warren wrote: Hi Rob, There is a fundamental problem with HTTP status codes. Lets say a user submits a complex but small sparql request. My server sees the syntax is good and starts to reply in good faith. This means the server starts the http response and sends an 200 O

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-19 Thread Samuel Rose
On Thu, Apr 18, 2013 at 9:23 AM, Andrea Splendiani wrote: > Hi, > > I think that some caching with a minimum of query rewriting would get read of > 90% of the select{?s ?p ?o} where {?s?p ?o} queries. > > From a user perspective, I would rather have a clear result code upfront > telling me: your

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-19 Thread Rob Warren
Hi Rob, There is a fundamental problem with HTTP status codes. Lets say a user submits a complex but small sparql request. My server sees the syntax is good and starts to reply in good faith. This means the server starts the http response and sends an 200 OK Some results are being send Howev

Re: Fwd: Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-19 Thread Kingsley Idehen
On 4/19/13 7:04 AM, Leigh Dodds wrote: Hi, On Fri, Apr 19, 2013 at 11:55 AM, Kingsley Idehen wrote: ... If you have OFFSET and LIMIT in use, you can reflect the new state of affairs when the next GET is performed i.e, lets say you have OFFSET 20 and LIMIT 20, the URL with OFFSET 40 is the requ

Re: Fwd: Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-19 Thread Leigh Dodds
Hi, On Fri, Apr 19, 2013 at 11:55 AM, Kingsley Idehen wrote: > ... > If you have OFFSET and LIMIT in use, you can reflect the new state of > affairs when the next GET is performed i.e, lets say you have OFFSET 20 and > LIMIT 20, the URL with OFFSET 40 is the request for the next batch of > result

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-19 Thread Kingsley Idehen
On 4/19/13 4:20 AM, Leigh Dodds wrote: Hi, On Fri, Apr 19, 2013 at 8:49 AM, Jerven Bolleman wrote: Original Message Subject: Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users. Date: Thu, 18 Apr 2013 23:21:46 +0200 From: Jerven Bolleman To: Rob

Re: Fwd: Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-19 Thread Kingsley Idehen
On 4/19/13 3:49 AM, Jerven Bolleman wrote: Forgot reply all Original Message Subject: Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users. Date: Thu, 18 Apr 2013 23:21:46 +0200 From: Jerven Bolleman To: Rob Warren Hi Rob, There is a

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-19 Thread Kingsley Idehen
On 4/18/13 4:53 PM, Rob Warren wrote: On 18-Apr-13, at 8:53 AM, Jerven Bolleman wrote: Many of the current public SPARQL endpoints limit all their users to queries of limited CPU time. But this is not enough to really manage (mis) use of an endpoint. Also the SPARQL api being http based suffe

Re: Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-19 Thread Leigh Dodds
Hi, On Fri, Apr 19, 2013 at 8:49 AM, Jerven Bolleman wrote: > Original Message > Subject: Re: Public SPARQL endpoints:managing (mis)-use and communicating > limits to users. > Date: Thu, 18 Apr 2013 23:21:46 +0200 > From: Jerven Bolleman > To: Rob Warren >

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-19 Thread Rob Warren
On 18-Apr-13, at 8:53 AM, Jerven Bolleman wrote: Many of the current public SPARQL endpoints limit all their users to queries of limited CPU time. But this is not enough to really manage (mis) use of an endpoint. Also the SPARQL api being http based suffers from the problem that we first sen

Fwd: Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-19 Thread Jerven Bolleman
Forgot reply all Original Message Subject: Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users. Date: Thu, 18 Apr 2013 23:21:46 +0200 From: Jerven Bolleman To: Rob Warren Hi Rob, There is a fundamental problem with HTTP status codes. Lets say

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-18 Thread Kingsley Idehen
On 4/18/13 7:06 PM, Andrea Splendiani wrote: Il giorno 18/apr/2013, alle ore 16:04, Kingsley Idehen ha scritto: On 4/18/13 9:23 AM, Andrea Splendiani wrote: Hi, I think that some caching with a minimum of query rewriting would get read of 90% of the select{?s ?p ?o} where {?s?p ?o} queries

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-18 Thread Andrea Splendiani
Il giorno 18/apr/2013, alle ore 16:04, Kingsley Idehen ha scritto: > On 4/18/13 9:23 AM, Andrea Splendiani wrote: >> Hi, >> >> I think that some caching with a minimum of query rewriting would get read >> of 90% of the select{?s ?p ?o} where {?s?p ?o} queries. > Sorta. > Client queries are inh

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-18 Thread Kingsley Idehen
On 4/18/13 12:07 PM, Alan Ruttenberg wrote: On Thu, Apr 18, 2013 at 11:55 AM, Jerven Bolleman mailto:jerven.bolle...@isb-sib.ch>> wrote: Hi Alan, On Apr 18, 2013, at 5:33 PM, Alan Ruttenberg wrote: > > On Thu, Apr 18, 2013 at 7:53 AM, Jerven Bolleman mailto:jerven.bolle

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-18 Thread Alan Ruttenberg
On Thu, Apr 18, 2013 at 11:55 AM, Jerven Bolleman < jerven.bolle...@isb-sib.ch> wrote: > Hi Alan, > On Apr 18, 2013, at 5:33 PM, Alan Ruttenberg wrote: > > > > > On Thu, Apr 18, 2013 at 7:53 AM, Jerven Bolleman < > jerven.bolle...@isb-sib.ch> wrote: > > >Last but not least how can we avoid that us

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-18 Thread Jerven Bolleman
Hi Alan, On Apr 18, 2013, at 5:33 PM, Alan Ruttenberg wrote: > > On Thu, Apr 18, 2013 at 7:53 AM, Jerven Bolleman > wrote: > >Last but not least how can we avoid that users need to run SELECT > >(COUNT(DISTINT(?s) as ?sc} WHERE {?s ?p ?o} and friends. > It's always rather disappointing to me t

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-18 Thread Alan Ruttenberg
On Thu, Apr 18, 2013 at 7:53 AM, Jerven Bolleman wrote: > Last but not least how can we avoid that users need to run SELECT > (COUNT(DISTINT(?s) as ?sc} WHERE {?s ?p ?o} and friends. I am interested in why queries like this are not optimized. Seems to me that this should be straightforward to o

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-18 Thread Kingsley Idehen
On 4/18/13 9:23 AM, Andrea Splendiani wrote: Hi, I think that some caching with a minimum of query rewriting would get read of 90% of the select{?s ?p ?o} where {?s?p ?o} queries. Sorta. Client queries are inherently unpredictable. That's always been the case, and that predates SPARQL. Thes

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-18 Thread Andrea Splendiani
Hi, Il giorno 18/apr/2013, alle ore 15:21, Jerven Bolleman ha scritto: >> I think that some caching with a minimum of query rewriting would get read >> of 90% of the select{?s ?p ?o} where {?s?p ?o} queries. > We have some caching on the uniprot side. But as all queries are nearly > unique res

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-18 Thread Jerven Bolleman
Hi All, On Apr 18, 2013, at 3:23 PM, Andrea Splendiani wrote: > Hi, > > I think that some caching with a minimum of query rewriting would get read of > 90% of the select{?s ?p ?o} where {?s?p ?o} queries. We have some caching on the uniprot side. But as all queries are nearly unique result cach

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-18 Thread Andrea Splendiani
Hi, I think that some caching with a minimum of query rewriting would get read of 90% of the select{?s ?p ?o} where {?s?p ?o} queries. From a user perspective, I would rather have a clear result code upfront telling me: your query is to heavy, not enough resources and so on, than partial resul

Re: Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-18 Thread Kingsley Idehen
On 4/18/13 7:53 AM, Jerven Bolleman wrote: Hi All, Managing a public SPARQL endpoint has some difficulties in comparison to managing a simpler REST api. Instead of counting api calls or external bandwidth use we need to look at internal IO and CPU usage as well. Many of the current public SPA

Public SPARQL endpoints:managing (mis)-use and communicating limits to users.

2013-04-18 Thread Jerven Bolleman
Hi All, Managing a public SPARQL endpoint has some difficulties in comparison to managing a simpler REST api. Instead of counting api calls or external bandwidth use we need to look at internal IO and CPU usage as well. Many of the current public SPARQL endpoints limit all their users to querie