Andy, Yes, I'm running the server and using the client with direct access to Fuseki (i.e. without proxy).
When the Exception occurs in client side, the server outputs to log: *...* *14:23:22 INFO Fuseki :: [443] 200 OK* *14:23:22 INFO Fuseki :: [444] GET http://fuseki.mooo.com:8080/dbpedia-geo/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FDonaldsonville%252C_Louisiana%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23lat%3E+%3Flat+.%0A++++%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FDonaldsonville%252C_Louisiana%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23long%3E+%3Flong%0A++%7D%0A * *14:23:22 INFO Fuseki :: [444] Query = SELECT * WHERE { < http://dbpedia.org/resource/Donaldsonville%2C_Louisiana> < http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat . < http://dbpedia.org/resource/Donaldsonville%2C_Louisiana> < http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long } * *14:23:22 INFO Fuseki :: [444] OK/select* *14:23:22 WARN Fuseki :: [444] RC = 500 : Direct buffer memory* When I use SERVICE SILENT, the client outputs: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 15:34:19 WARN QueryIterService :: SERVICE: HttpException: 500 Server Error 15:34:20 WARN QueryIterService :: SERVICE: HttpException: 500 Server Error 15:34:21 WARN QueryIterService :: SERVICE: HttpException: 500 Server Error 15:34:21 WARN QueryIterService :: SERVICE: HttpException: 500 Server Error 15:34:22 WARN QueryIterService :: SERVICE: HttpException: 500 Server Error 15:34:23 WARN QueryIterService :: SERVICE: HttpException: 500 Server Error 15:34:23 WARN QueryIterService :: SERVICE: HttpException: 500 Server Error 15:34:24 WARN QueryIterService :: SERVICE: HttpException: 500 Server Error 15:34:25 WARN QueryIterService :: SERVICE: HttpException: 500 Server Error ... How can I avoid "Direct buffer memory" problem? The client uses *jena-arq-2.9.0-incubating.jar* and the server uses * Fuseki-0.2.1-incubating-SNAPSHOT*. Regards, Regis. On Mon, Apr 9, 2012 at 1:17 PM, Andy Seaborne <[email protected]> wrote: > Regis, > > What is fuseki.mooo.com? Are you running that server? > > 500 means the server is unhappy -- the server log file may give further > details. > > I don't know why you are seeing large numbers of connections - the client > code uses vanilla java.net HTTP code so it should reuse connections or > close them (at least in the ARQ 2.9.0). > > I'm not familiar with NginX as a proxy. ARQ does try to execute more > specific SERVICE requests that use one vague one - sometimes the right > thing to do, sometimes not. The engine can't tell. > > We did recently (post 2.9.0 release - available in 2.9.1-SNAPSHOT) change > the SERVICE handling code. If you say "SERVICE SILENT" it will read all > the results from a call before moving on - that might help. > > Andy > > > On 08/04/12 14:00, Regis Pires Magalhães wrote: > >> This code can be used to see the HttpException (500 Server Error) remotely >> (only 18 results found before the exception occurs): >> >> import com.hp.hpl.jena.query.**QueryExecution; >> import com.hp.hpl.jena.query.**QueryExecutionFactory; >> import com.hp.hpl.jena.query.**QueryFactory; >> import com.hp.hpl.jena.query.**QuerySolution; >> import com.hp.hpl.jena.query.**ResultSet; >> import com.hp.hpl.jena.rdf.model.**ModelFactory; >> import com.hp.hpl.jena.sparql.core.**DatasetImpl; >> >> public class Query { >> public static void main(String[] args) { >> String query = "PREFIX owl:<http://www.w3.org/2002/** >> 07/owl# <http://www.w3.org/2002/07/owl#>>\n" >> + >> "PREFIX geopos:< >> http://www.w3.org/2003/01/geo/**wgs84_pos#<http://www.w3.org/2003/01/geo/wgs84_pos#>>\n" >> + >> "select ?s ?lat ?long where {\n" + >> " SERVICE >> <http://*fuseki.mooo.com:8080***/lgd-dbpedia/sparql> >> >> {\n" + >> " select * where {\n" + >> " ?s owl:sameAs ?geo .\n" + >> " FILTER regex(str(?geo),\"dbpedia\")\ >> **n" >> + >> " } \n" + >> " }\n" + >> " SERVICE >> <http://*fuseki.mooo.com:8080***/dbpedia-geo/sparql> >> >> {\n" + >> " select * where{\n" + >> " ?geo geopos:lat ?lat ;\n" + >> " geopos:long ?long .\n" + >> " } \n" + >> " } \n" + >> "} \n"; >> >> long start = System.currentTimeMillis(); >> QueryExecution exec = >> QueryExecutionFactory.create(**QueryFactory.create(query), >> new >> DatasetImpl(ModelFactory.**createDefaultModel())); >> ResultSet rs = exec.execSelect(); >> int n = 0; >> while (rs.hasNext()) { >> rs.next(); >> System.out.print(++n + " "); >> } >> long time = System.currentTimeMillis() - start; >> System.out.println(n + " results found in " + time + >> "ms."); >> } >> } >> >> >> Abraços, >> Regis. >> >> >> 2012/4/8 Regis Pires >> Magalhães<regispiresmag@gmail.**com<[email protected]> >> > >> >> Andy, >>> The exception is always the same NoRouteToHostException and we the only >>> proxy we use between the client and the Fuseki server is a reverse proxy >>> (NginX). >>> >>> I've also observed that the number of network connections with the server >>> increases from 0 to a bit more than 28000, when the exception >>> occurs. Almost all of the connections are TIME_WAIT. >>> >>> When I try a direct connection to Fuseki (even locally using 127.0.0.1 as >>> the address to the Sparql Endpoint) I only get 17 results (the number of >>> connections increases from 0 to 212) and the exception changes to: >>> >>> Exception in thread "main" HttpException: HttpException: 500 Server Error >>> : HttpException: 500 Server Error >>> at >>> com.hp.hpl.jena.sparql.engine.**http.HttpQuery.execCommon(** >>> HttpQuery.java:375) >>> at com.hp.hpl.jena.sparql.engine.**http.HttpQuery.execGet(** >>> HttpQuery.java:202) >>> at com.hp.hpl.jena.sparql.engine.**http.HttpQuery.exec(HttpQuery.** >>> java:157) >>> at com.hp.hpl.jena.sparql.engine.**http.Service.exec(Service.**java:71) >>> at >>> com.hp.hpl.jena.sparql.engine.**main.iterator.** >>> QueryIterService.nextStage(**QueryIterService.java:53) >>> at >>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterRepeatApply.** >>> makeNextStage(**QueryIterRepeatApply.java:113) >>> at >>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterRepeatApply.** >>> hasNextBinding(**QueryIterRepeatApply.java:65) >>> at >>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.** >>> hasNext(QueryIteratorBase.**java:108) >>> at >>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterConvert.** >>> hasNextBinding(**QueryIterConvert.java:65) >>> at >>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.** >>> hasNext(QueryIteratorBase.**java:108) >>> at >>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorWrapper.** >>> hasNextBinding(**QueryIteratorWrapper.java:40) >>> at >>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.** >>> hasNext(QueryIteratorBase.**java:108) >>> at >>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorWrapper.** >>> hasNextBinding(**QueryIteratorWrapper.java:40) >>> at >>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.** >>> hasNext(QueryIteratorBase.**java:108) >>> at >>> com.hp.hpl.jena.sparql.engine.**ResultSetStream.hasNext(** >>> ResultSetStream.java:72) >>> at Query.main(Query.java:42) >>> Caused by: HttpException: 500 Server Error >>> at >>> com.hp.hpl.jena.sparql.engine.**http.HttpQuery.execCommon(** >>> HttpQuery.java:320) >>> ... 15 more >>> >>> How could I get all the results? >>> >>> Regards, >>> Regis. >>> >>> >>> >>> On Sun, Apr 8, 2012 at 7:38 AM, Andy Seaborne<[email protected]> wrote: >>> >>> On 08/04/12 01:30, Regis Pires Magalhăes wrote: >>>> >>>> I've got a sucessful result when executing *outside* our intranet >>>>> (LAN). >>>>> >>>>> All the 43016 results were returned. >>>>> It seems that there are so many requests (when executing in the Local >>>>> Network) that the server or the client can not properly >>>>> handle them. Suggestions? I could not receive more than 5000 results >>>>> when >>>>> querying inside the LAN. Results come much faster, but the exception >>>>> always >>>>> occurs. >>>>> >>>>> Regis. >>>>> >>>>> >>>> Is it the same exception? NoRouteToHostException isn't usually a sign >>>> of >>>> overload - it's a sign that the JVM inside the firewall can't contact >>>> the >>>> machine outside. Your proxy may be doing funny stuff though. >>>> >>>> Do you have proxy settings given to the JVM?it should work. >>>> >>>> -Dhttp.proxyHost=... >>>> -Dhttp.proxyPort=... >>>> >>>> or >>>> -DsocksProxyHost=... >>>> -DsocksProxyPort=... >>>> >>>> Andy >>>> >>>> >>>> >>>>> 2012/4/7 Regis Pires Magalhăes<regispiresmag@gmail.****com< >>>>> [email protected]> >>>>> >>>>>> >>>>>> >>>>> >>>>> I get a "java.net.****NoRouteToHostException: Cannot assign requested >>>>>> >>>>>> address" >>>>>> exception when I execute que following code: >>>>>> >>>>>> import com.hp.hpl.jena.query.****QueryExecution; >>>>>> import com.hp.hpl.jena.query.****QueryExecutionFactory; >>>>>> import com.hp.hpl.jena.query.****QueryFactory; >>>>>> import com.hp.hpl.jena.query.****QuerySolution; >>>>>> import com.hp.hpl.jena.query.****ResultSet; >>>>>> import com.hp.hpl.jena.rdf.model.****ModelFactory; >>>>>> import com.hp.hpl.jena.sparql.core.****DatasetImpl; >>>>>> >>>>>> >>>>>> public class Query { >>>>>> public static void main(String[] args) { >>>>>> String query = "PREFIX owl:< >>>>>> http://www.w3.org/2002/07/owl#**** <http://www.w3.org/2002/07/owl#**>>\n" >>>>>> + >>>>>> "PREFIX geopos:< >>>>>> http://www.w3.org/2003/01/geo/****wgs84_pos#<http://www.w3.org/2003/01/geo/**wgs84_pos#> >>>>>> <http://www.w3.**org/2003/01/geo/wgs84_pos#<http://www.w3.org/2003/01/geo/wgs84_pos#> >>>>>> >>\**n" >>>>>> >>>>>> + >>>>>> "select ?s ?lat ?long where {\n" + >>>>>> " SERVICE< >>>>>> http://fuseki.mooo.com/lgd-****dbpedia/sparql<http://fuseki.mooo.com/lgd-**dbpedia/sparql> >>>>>> <http://fuseki.**mooo.com/lgd-dbpedia/sparql<http://fuseki.mooo.com/lgd-dbpedia/sparql> >>>>>> >> >>>>>> >>>>>> {\n" + >>>>>> " select * where {\n" + >>>>>> " ?s owl:sameAs ?geo .\n" + >>>>>> " FILTER >>>>>> regex(str(?geo),\"dbpedia\")\****n" + >>>>>> >>>>>> " } \n" + >>>>>> " }\n" + >>>>>> " SERVICE< >>>>>> http://fuseki.mooo.com/****dbpedia-geo/sparql<http://fuseki.mooo.com/**dbpedia-geo/sparql> >>>>>> <http://**fuseki.mooo.com/dbpedia-geo/**sparql<http://fuseki.mooo.com/dbpedia-geo/sparql> >>>>>> >> >>>>>> >>>>>> {\n" + >>>>>> " select * where{\n" + >>>>>> " ?geo geopos:lat ?lat ;\n" + >>>>>> " geopos:long ?long .\n" + >>>>>> " } \n" + >>>>>> " } \n" + >>>>>> "} \n"; >>>>>> >>>>>> long start = System.currentTimeMillis(); >>>>>> QueryExecution exec = >>>>>> QueryExecutionFactory.create(****QueryFactory.create(query), >>>>>> new >>>>>> DatasetImpl(ModelFactory.****createDefaultModel())); >>>>>> >>>>>> ResultSet rs = exec.execSelect(); >>>>>> int n = 0; >>>>>> while (rs.hasNext()) { >>>>>> rs.next(); >>>>>> System.out.print(++n + " "); >>>>>> } >>>>>> long time = System.currentTimeMillis() - start; >>>>>> System.out.println(n + " results found in " + time + >>>>>> "ms."); >>>>>> } >>>>>> } >>>>>> >>>>>> The same query runs perfectly when a do limit in the first SERVICE >>>>>> select: >>>>>> >>>>>> String query = "PREFIX owl:< >>>>>> http://www.w3.org/2002/07/owl#**** <http://www.w3.org/2002/07/owl#**>>\n" >>>>>> + >>>>>> "PREFIX geopos:< >>>>>> http://www.w3.org/2003/01/geo/****wgs84_pos#<http://www.w3.org/2003/01/geo/**wgs84_pos#> >>>>>> <http://www.w3.**org/2003/01/geo/wgs84_pos#<http://www.w3.org/2003/01/geo/wgs84_pos#> >>>>>> >>\**n" >>>>>> >>>>>> + >>>>>> "select ?s ?lat ?long where {\n" + >>>>>> " SERVICE< >>>>>> http://fuseki.mooo.com/lgd-****dbpedia/sparql<http://fuseki.mooo.com/lgd-**dbpedia/sparql> >>>>>> <http://fuseki.**mooo.com/lgd-dbpedia/sparql<http://fuseki.mooo.com/lgd-dbpedia/sparql> >>>>>> >> >>>>>> >>>>>> {\n" + >>>>>> " select * where {\n" + >>>>>> " ?s owl:sameAs ?geo .\n" + >>>>>> " FILTER >>>>>> regex(str(?geo),\"dbpedia\")\****n" + >>>>>> >>>>>> " } *limit 20000* \n" + >>>>>> >>>>>> " }\n" + >>>>>> " service< >>>>>> http://fuseki.mooo.com/****dbpedia-geo/sparql<http://fuseki.mooo.com/**dbpedia-geo/sparql> >>>>>> <http://**fuseki.mooo.com/dbpedia-geo/**sparql<http://fuseki.mooo.com/dbpedia-geo/sparql> >>>>>> >> >>>>>> >>>>>> {\n" + >>>>>> " select * where{\n" + >>>>>> " ?geo geopos:lat ?lat ;\n" + >>>>>> " geopos:long ?long .\n" + >>>>>> " } \n" + >>>>>> " } \n" + >>>>>> "} \n"; >>>>>> >>>>>> >>>>>> What can I do to fix the problem? It always occurs when the query is >>>>>> run, >>>>>> even when the Fuseki Sparql Endpoint is running locally. >>>>>> I did not observe any problem related to the network. The problem >>>>>> persists >>>>>> when the code (query) is executed in different hosts. >>>>>> I am using Jena classes from "fuseki-server.jar" in the client >>>>>> classpath. >>>>>> I've also tested different Fuseki jar versions: 0.2.1, 0.2.2. >>>>>> The full stack trace is this: >>>>>> >>>>>> Exception in thread "main" HttpException: >>>>>> java.net.****NoRouteToHostException: Cannot assign requested address: >>>>>> java.net.****NoRouteToHostException: Cannot assign requested address >>>>>> at >>>>>> com.hp.hpl.jena.sparql.engine.****http.HttpQuery.execGet(** >>>>>> HttpQuery.java:216) >>>>>> at com.hp.hpl.jena.sparql.engine.****http.HttpQuery.exec(** >>>>>> HttpQuery.** >>>>>> java:157) >>>>>> at com.hp.hpl.jena.sparql.engine.****http.Service.exec(Service.** >>>>>> java:71) >>>>>> at >>>>>> com.hp.hpl.jena.sparql.engine.****main.iterator.** >>>>>> QueryIterService.nextStage(****QueryIterService.java:53) >>>>>> at >>>>>> com.hp.hpl.jena.sparql.engine.****iterator.**QueryIterRepeatApply.** >>>>>> makeNextStage(****QueryIterRepeatApply.java:113) >>>>>> at >>>>>> com.hp.hpl.jena.sparql.engine.****iterator.**QueryIterRepeatApply.** >>>>>> hasNextBinding(****QueryIterRepeatApply.java:65) >>>>>> at >>>>>> com.hp.hpl.jena.sparql.engine.****iterator.QueryIteratorBase.**** >>>>>> hasNext(QueryIteratorBase.****java:108) >>>>>> at >>>>>> com.hp.hpl.jena.sparql.engine.****iterator.QueryIterConvert.** >>>>>> hasNextBinding(****QueryIterConvert.java:65) >>>>>> at >>>>>> com.hp.hpl.jena.sparql.engine.****iterator.QueryIteratorBase.**** >>>>>> hasNext(QueryIteratorBase.****java:108) >>>>>> at >>>>>> com.hp.hpl.jena.sparql.engine.****iterator.**QueryIteratorWrapper.** >>>>>> hasNextBinding(****QueryIteratorWrapper.java:40) >>>>>> at >>>>>> com.hp.hpl.jena.sparql.engine.****iterator.QueryIteratorBase.**** >>>>>> hasNext(QueryIteratorBase.****java:108) >>>>>> at >>>>>> com.hp.hpl.jena.sparql.engine.****iterator.**QueryIteratorWrapper.** >>>>>> hasNextBinding(****QueryIteratorWrapper.java:40) >>>>>> at >>>>>> com.hp.hpl.jena.sparql.engine.****iterator.QueryIteratorBase.**** >>>>>> hasNext(QueryIteratorBase.****java:108) >>>>>> at >>>>>> com.hp.hpl.jena.sparql.engine.****ResultSetStream.hasNext(** >>>>>> >>>>>> ResultSetStream.java:72) >>>>>> at Query.main(Query.java:34) >>>>>> Caused by: java.net.****NoRouteToHostException: Cannot assign >>>>>> requested >>>>>> address >>>>>> at java.net.PlainSocketImpl.****socketConnect(Native Method) >>>>>> at >>>>>> java.net.****AbstractPlainSocketImpl.****doConnect(** >>>>>> AbstractPlainSocketImpl.java:****327) >>>>>> at >>>>>> java.net.****AbstractPlainSocketImpl.****connectToAddress(** >>>>>> AbstractPlainSocketImpl.java:****193) >>>>>> at >>>>>> java.net.****AbstractPlainSocketImpl.****connect(** >>>>>> AbstractPlainSocketImpl.java:****180) >>>>>> at java.net.SocksSocketImpl.****connect(SocksSocketImpl.java:****384) >>>>>> at java.net.Socket.connect(****Socket.java:546) >>>>>> at java.net.Socket.connect(****Socket.java:495) >>>>>> at sun.net.NetworkClient.****doConnect(NetworkClient.java:****178) >>>>>> at sun.net.www.http.HttpClient.****openServer(HttpClient.java:**** >>>>>> 409) >>>>>> at sun.net.www.http.HttpClient.****openServer(HttpClient.java:**** >>>>>> 530) >>>>>> at sun.net.www.http.HttpClient.<****init>(HttpClient.java:240) >>>>>> at sun.net.www.http.HttpClient.****New(HttpClient.java:321) >>>>>> at sun.net.www.http.HttpClient.****New(HttpClient.java:338) >>>>>> at >>>>>> sun.net.www.protocol.http.****HttpURLConnection.**** >>>>>> getNewHttpClient(** >>>>>> HttpURLConnection.java:935) >>>>>> at >>>>>> sun.net.www.protocol.http.****HttpURLConnection.****plainConnect(** >>>>>> HttpURLConnection.java:876) >>>>>> at >>>>>> sun.net.www.protocol.http.****HttpURLConnection.connect(** >>>>>> HttpURLConnection.java:801) >>>>>> at >>>>>> com.hp.hpl.jena.sparql.engine.****http.HttpQuery.execGet(** >>>>>> >>>>>> HttpQuery.java:199) >>>>>> ... 14 more >>>>>> >>>>>> Best regards, >>>>>> Regis. >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >
