Andy, I did a local profiling of the query in a standard Eclipse/YourKit configuration which took 45s. looks like this is just a matter of increasing heap space to allow fuseki to complete the query now.
(slice 0 10000 (project (?r ?count) (extend ((?count ?.0)) (group (?s ?r) ((?.0 (count))) (join (bgp (triple ?x ?r ?s)) (slice 124639 1000 (project (?s) (bgp (triple ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?o))))))))) On Wed, Dec 19, 2018 at 10:57 AM Marco Neumann <marco.neum...@gmail.com> wrote: > > so yes an apache proxy and a query timeout limit for fuseki instances > it will be. > > I just checked the same query on an open source virtuoso instance > (7.2) with the same data and it seems that virtuoso handles the > request much more resourcefully and to completion. Andy can you > enlighten me what the main difference here is in the treatment of the > query by jena (39s) vs virtuoso (1s)? > > On Wed, Dec 19, 2018 at 6:56 AM Laura Morales <laure...@mail.com> wrote: > > > > > and needs some explaining why we put open endpoints on the web without > > > great restrictions > > > > I've always been puzzled by this as well. You never see a publicly > > reachable PostgreSQL or MariaDB servers, or any other database. There is > > always a layer in between which defines a list of possible requests, and > > then every requests is optimized to retrieve data from the database. With a > > public endpoint instead, this optimization is not possible since anybody > > can write any query. I think the reason is simply that a sparql endpoint is > > supposed to answer any type of query which traverses any path that is not > > well defined a priori. If you only want the server to serve a specific kind > > of queries instead, in this case you can in fact use some kind of REST API > > in front of it and translate every request to a sparql query; in this > > scenario you don't need the endpoint to be public, but you're limiting the > > type of queries that a user can ask. > > > > > > > > > > Sent: Tuesday, December 18, 2018 at 11:40 PM > > From: "Marco Neumann" <marco.neum...@gmail.com> > > To: "Bruno P. Kinoshita" <brunodepau...@yahoo.com.br>, users@jena.apache.org > > Subject: Re: blocking IP to prevent malicious sparql queries > > It's good to see people using sparql one way or another. It's still an > > unusual thing in the wild and needs some explaining why we put open > > endpoints on the web without great restrictions. But since this one is > > intended to be a sandbox to play with and learn I take indeed a positive > > view on this incident. > > > > On Tue 18 Dec 2018 at 21:34, Bruno P. Kinoshita > > <brunodepau...@yahoo.com.br.invalid> wrote: > > > > > I think Laura's option is the best/easiest one, and good on you for the > > > positive point-of-view on these spams Marco! :D > > > Bruno > > > > > > From: Marco Neumann <marco.neum...@gmail.com> > > > To: users@jena.apache.org > > > Sent: Wednesday, 19 December 2018 8:58 AM > > > Subject: Re: blocking IP to prevent malicious sparql queries > > > > > > Thank you Laura, > > > > > > I was hoping for a quick fix and something along the lines of a fuseki > > > blacklist filter in the shiro.ini > > > > > > but yes the reverse proxy is probably a more sensible approach at this > > > point. > > > > > > In any event good to see sparql spam like this here, it means that the > > > Semantic Web has most certainly arrived in the mainstream ;) > > > > > > > > > > > > On Tue, Dec 18, 2018 at 5:35 PM Laura Morales <laure...@mail.com> wrote: > > > > > > > While I think the correct answer is YES (perhaps by implementing a > > > > custom > > > > filter), I guess the answer is going to be "use a reverse proxy". > > > > > > > > > > > > > > > > > > > > Sent: Tuesday, December 18, 2018 at 6:16 PM > > > > From: "Marco Neumann" <marco.neum...@gmail.com> > > > > To: users@jena.apache.org > > > > Subject: blocking IP to prevent malicious sparql queries > > > > is it possible to block indiviual IPs with the shiro.ini? > > > > > > > > We receive a number of malicious sparql queries from an IP in France > > > > (193.52.210.70) today > > > > > > > > that continuously issues the following SPARQL query: > > > > > > > > SELECT ?r (count(*) AS ?count) > > > > WHERE{ ?x ?r ?s > > > > { SELECT ?s WHERE > > > > { ?s a ?o } > > > > OFFSET 124639 LIMIT 1000 } > > > > } GROUP BY ?s ?r OFFSET 0 LIMIT 10000 > > > > > > > > resulting in: > > > > > > > > [2018-12-18 18:10:31] AbstractConnector WARN > > > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > > > [2018-12-18 18:10:34] Fuseki WARN [424] RC = 500 : GC overhead limit > > > > exceeded > > > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > > > [2018-12-18 18:10:34] Fuseki INFO [424] 500 GC overhead limit exceeded > > > > (39.946 s) > > > > > > > > and pushes fuseki offline for a few minutes. > > > > > > > > > > > > -- > > > > > > > > > > > > --- > > > > Marco Neumann > > > > KONA > > > > > > > > > > > > > -- > > > > > > > > > --- > > > Marco Neumann > > > KONA > > > > > > > > > > > > > -- > > > > > > --- > > Marco Neumann > > KONA > > > > -- > > > --- > Marco Neumann > KONA -- --- Marco Neumann KONA