Re: Fuseki errors with concurrent requests

Hi, is it somehow possible to get this error (TransactionManager ERROR There are now active transactions ) from Fuseki GSP so that client could sleep and retry later allowing db to finish the transaction? On 19.6.2018 23:56, Andy Seaborne wrote: On 18/06/18 11:36, Mikael Pesonen wrote:

Re: Fuseki errors with concurrent requests

Hi, On 19.6.2018 23:56, Andy Seaborne wrote: >>> errors occur less than before. which errors?    TransactionManager ERROR There are now active transactions This occurs when Im inserting new data and trying to read it right away (get empty result), or deleting data and trying to read it to

Re: Fuseki errors with concurrent requests

On 18/06/18 11:36, Mikael Pesonen wrote: On 18.6.2018 13:30, Andy Seaborne wrote: On 18/06/18 10:55, Mikael Pesonen wrote: Hi Andy, On 18.6.2018 12:05, Andy Seaborne wrote: >>> errors occur less than before. which errors?   TransactionManager ERROR There are now active transacti

Re: Fuseki errors with concurrent requests

On 18.6.2018 13:30, Andy Seaborne wrote: On 18/06/18 10:55, Mikael Pesonen wrote: Hi Andy, On 18.6.2018 12:05, Andy Seaborne wrote: On 15/06/18 15:36, Mikael Pesonen wrote: Hi, unfortunately I haven't been able to make a standalone package yet. However with jena 3.7.0 things are bi

Re: Fuseki errors with concurrent requests

On 18/06/18 10:55, Mikael Pesonen wrote: Hi Andy, On 18.6.2018 12:05, Andy Seaborne wrote: On 15/06/18 15:36, Mikael Pesonen wrote: Hi, unfortunately I haven't been able to make a standalone package yet. However with jena 3.7.0 things are bit better now - errors occur less than befor

Re: Fuseki errors with concurrent requests

Hi Andy, On 18.6.2018 12:05, Andy Seaborne wrote: On 15/06/18 15:36, Mikael Pesonen wrote: Hi, unfortunately I haven't been able to make a standalone package yet. However with jena 3.7.0 things are bit better now - errors occur less than before. "The errors" are which exactly? the thr

Re: Fuseki errors with concurrent requests

On 15/06/18 15:36, Mikael Pesonen wrote: Hi, unfortunately I haven't been able to make a standalone package yet. However with jena 3.7.0 things are bit better now - errors occur less than before. "The errors" are which exactly? the thread doesn't make it clear. Actually now I'm not gett

Re: Fuseki errors with concurrent requests

Hi, unfortunately I haven't been able to make a standalone package yet. However with jena 3.7.0 things are bit better now - errors occur less than before. Actually now I'm not getting error codes from Fuseki (all 200), but errors still occur and at that time there is Fuseki output  Trans

Re: Fuseki errors with concurrent requests

On 07/03/18 12:44, Mikael Pesonen wrote: So can we make backup quads, empty database and insert quads back, and then we have a noncorrupted database? It can't be guaranteed. If the quad dump is valid RDF, the new database will not be corrupted. But a dump of a broken database may be incom

Re: Fuseki errors with concurrent requests

So can we make backup quads, empty database and insert quads back, and then we have a noncorrupted database? On 5.3.2018 16:41, ajs6f wrote: To my knowledge (Andy of course is the TDB expert) you can't really rebuild a TDB instance from a corrupted TDB instance. You should start with a know

Re: Fuseki errors with concurrent requests

Thanks for the tip. Out test has so much logic, testing of search results, comparing stuff etc, so we decided to build it with PHP. But from Jmeter got an idea, is it possible to save all calls to Jena (with content, headers) so that it's possible to rerun the test without our environment?

Re: Fuseki errors with concurrent requests

Maybe you can make a reproducible using JMeter or such. On Tue, Mar 6, 2018 at 11:24 AM, Mikael Pesonen wrote: > > Yes, clean install of Ubuntu, Jena etc. > > > > > On 5.3.2018 17:40, Andy Seaborne wrote: > >> >> >> On 05/03/18 15:04, Mikael Pesonen wrote: >> >>> >>> We are using GSP and our tes

Re: Fuseki errors with concurrent requests

Yes, clean install of Ubuntu, Jena etc. On 5.3.2018 17:40, Andy Seaborne wrote: On 05/03/18 15:04, Mikael Pesonen wrote: We are using GSP and our test script is doing ~20 json-ld inserts and sparql updates in a row ASAP, and we are running 10 test scripts concurrently. This test is fail

Re: Fuseki errors with concurrent requests

On 05/03/18 15:04, Mikael Pesonen wrote: We are using GSP and our test script is doing ~20 json-ld inserts and sparql updates in a row ASAP, and we are running 10 test scripts concurrently. This test is failing now. Starting with an empty database? On 5.3.2018 16:51, ajs6f wrote: "fai

Re: Fuseki errors with concurrent requests

From https://jena.apache.org/documentation/fuseki2/fuseki-server-protocol.html#backup : > Backups are written to the server local directory 'backups' as > gzip-compressed N-Quads files. You can load them just like any other NQuads files. ajs6f > On Mar 5, 2018, at 9:54 AM, Mikael Pesonen wr

Re: Fuseki errors with concurrent requests

We are using GSP and our test script is doing ~20 json-ld inserts and sparql updates in a row ASAP, and we are running 10 test scripts concurrently. This test is failing now. On 5.3.2018 16:51, ajs6f wrote: "fairly high load and concurrent usage" This is not a very precise or reproducible

Re: Fuseki errors with concurrent requests

You can't reliably copy the database files while the server s running. To backup, dump in Trig or N-Quads : "GET http://HOST/DATABASE"; Andy On 05/03/18 14:54, Mikael Pesonen wrote: I understood that content has to be rebuilt somehow, just copying files from backup is not enough? And

Re: Fuseki errors with concurrent requests

I understood that content has to be rebuilt somehow, just copying files from backup is not enough? And how do we know how old backup is needed, e.g. when the corruption did happen? On 5.3.2018 16:50, ajs6f wrote: I don't understand-- if you have known-good TDB backups available, why would

Re: Fuseki errors with concurrent requests

"fairly high load and concurrent usage" This is not a very precise or reproducible measure. Many sites use Jena in production at all kinds of scales for all kinds of dimensions, including HA setups. If you can explain more about your specific situation, you will get more useful advice. ajs6f

Re: Fuseki errors with concurrent requests

I don't understand-- if you have known-good TDB backups available, why would you not start with them? Dumping RDF files on the side is not a bad idea either, but TDB backups (such as are produced by the software itself) should be fine for most disaster-recovery purposes. ajs6f > On Mar 5, 201

Re: Fuseki errors with concurrent requests

Fortunately this is our development site. But backups are from TDP files, we dont have a plan for dumping RDF exports for separate backup. So I guess we should add that as next step? On 5.3.2018 16:45, ajs6f wrote: Not a problem at all. Many sites use TDB as their main store. Just like _any_

Re: Fuseki errors with concurrent requests

To be clear: can Jena be recommended for production database in our customer cases for fairly high load and concurrent usage? Or is it mainly for scientific purposes? Br On 5.3.2018 16:41, ajs6f wrote: To my knowledge (Andy of course is the TDB expert) you can't really rebuild a TDB instan

Re: Fuseki errors with concurrent requests

Not a problem at all. Many sites use TDB as their main store. Just like _any_ database, proper operation includes regular and frequent backups and a plan for rebuilding the store independently of any instance. You _do_ have backups, right? ajs6f > On Mar 5, 2018, at 9:43 AM, Mikael Pesonen w

Re: Fuseki errors with concurrent requests

Hi, we are using Jena TDP as THE database for document metadata. Data is fed through our custom REST API. Is this something that would no be recommended? Br, On 5.3.2018 16:41, ajs6f wrote: To my knowledge (Andy of course is the TDB expert) you can't really rebuild a TDB instance from a c

Re: Fuseki errors with concurrent requests

To my knowledge (Andy of course is the TDB expert) you can't really rebuild a TDB instance from a corrupted TDB instance. You should start with a known-good backup or original RDF files. ajs6f > On Mar 5, 2018, at 9:32 AM, Mikael Pesonen wrote: > > > Still having these issues on all of our i

Re: Fuseki errors with concurrent requests

Still having these issues on all of our installations. I'm going to rule out corrupted database on our oldest server. What would be preferred way to rebuild data? Data folder:  5226102784 Mar  5 12:48 GOSP.dat   260046848 Mar  5 12:48 GOSP.idn  5377097728 Mar  5 12:48 GPOS.dat   268435456 Ma

Re: Fuseki errors with concurrent requests

Hi, seems that building an isolated test setup for reproducing the issue is difficult. Would it be possible/helpful if we arrange an access to our server and the test script which does generate the error? Mikael On 30.1.2018 15:04, Andy Seaborne wrote: These seem to be different errors.

Re: Fuseki errors with concurrent requests

I can prepare one. So to be clear, when using Apache web server -> multiple php processes -> Fuseki/GSP over HTTP, is that safe for read and write? Or does this configuration need an additional single queue for write operations? On 30.1.2018 15:00, Andy Seaborne wrote: Could you please tur

Re: Fuseki errors with concurrent requests

On 29/01/18 14:56, ajs6f wrote: TDB1 is MRSW (multiple reader or single writer) TDB1 is multiple reader and single writer > it's difficult to say much without a reproducible phenomenon Yes - I can't see any specific clues here so far. Andy

Re: Fuseki errors with concurrent requests

These seem to be different errors. "In the middle of an alloc-write" is possibly a concurrency issue. "Failed to read" is possibly a previous corrupted database This is a text dataset? That should be using an MRSW lock to get some level isolation. What's the Fuseki config in this case? A

Re: Fuseki errors with concurrent requests

Could you please turn this in to a standalone complete, minimal example? AKA something with all the details that can be run by someone else including how the server is being run, what disk storage you are using, and whether the database starts from fresh or not. Does it happen on earlier vers

Re: Fuseki errors with concurrent requests

Hi, My test is dependent of REST API we developed. So basically simultaneous calls to Apahe web server which loads php which calls Fuseki using curl. On 29.1.2018 16:56, ajs6f wrote: That might be worth trying, although since TDB1 is MRSW (multiple reader or single writer), that queuing o

Re: Fuseki errors with concurrent requests

Yes the loading is single-threaded. Its a simple app that performs bulk loading using: DatasetAccessorFactory.createHTTP(baseUrl+"/data”); and for the first model to transfer: DatasetAccessor putModel(graphName, m); and for following models: static void addToTransferBulk(final St

Re: Fuseki errors with concurrent requests

That (using a queue) would depend on what you mean by "a database load going on with our own app". Are you doing those updates singlethreaded? ajs6f > On Jan 29, 2018, at 10:27 AM, Chris Tomlinson > wrote: > > I don’t have any test code per se. I was running a load tool that we wrote > (code

Re: Fuseki errors with concurrent requests

I don’t have any test code per se. I was running a load tool that we wrote (code snippets included) and issuing a few simple sparql’s via the fuseki browser app. That’s the extent of the test harness. I don’t see how the set up that I described would make use of a queue. Chris > On Jan 29, 20

Re: Fuseki errors with concurrent requests

That might be worth trying, although since TDB1 is MRSW (multiple reader or single writer), that queuing of updates should be going on on the server-side. I haven't had time to look at this issue, and it's difficult to say much without a reproducible phenomenon. Do you either of y'all have test

Re: Fuseki errors with concurrent requests

Until better solution, quick one would be to put all operations through a single queue? Br On 25.1.2018 4:11, Chris Tomlinson wrote: Also, Here's a link to the fuseki config: https://raw.githubusercontent.com/BuddhistDigitalResourceCenter/buda-base/master/conf/fuseki/bdrc-example.ttl Chri

Re: Fuseki errors with concurrent requests

Also, Here's a link to the fuseki config: https://raw.githubusercontent.com/BuddhistDigitalResourceCenter/buda-base/master/conf/fuseki/bdrc-example.ttl Chris > On Jan 24, 2018, at 17:40, Chris Tomlinson > wrote: > > On the latest 3.7.0-Snapshot (master branch) I also saw repeated occurrences

Re: Fuseki errors with concurrent requests

On the latest 3.7.0-Snapshot (master branch) I also saw repeated occurrences of this the other day while running some queries from the fuseki browser app and with a database load going on with our own app using: DatasetAccessorFactory.createHTTP(baseUrl+"/data”); with for the first model t

Re: Fuseki errors with concurrent requests

And running the test set with 1 concurrent loop, it was repeated 1 times without errors. So error occurs only with more than one concurrent operations. On 24.1.2018 17:40, ajs6f wrote: Please show your code, including Fuseki config. It's hard to diagnose what is going wrong when we don'

Re: Fuseki errors with concurrent requests

Hi, query is posted with curl to :3030/ds This is the error message: SELECT DISTINCT ?s ?p ?o WHERE {   GRAPH {     { ?s dcterms:isFormatOf }     UNION     {

Re: Fuseki errors with concurrent requests

Please show your code, including Fuseki config. It's hard to diagnose what is going wrong when we don't know what is supposed to happen. ajs6f > On Jan 24, 2018, at 10:26 AM, Mikael Pesonen > wrote: > > > Hi, > > I have a test script running 10 concurrent CRUD operations in a loop. After >

Fuseki errors with concurrent requests

Hi, I have a test script running 10 concurrent CRUD operations in a loop. After few operations I get HTTP error 500 and these are in fuseki server output. Am I doing something wrong here? [2018-01-24 17:16:53] BindingTDB ERROR get1(?o) org.apache.jena.tdb.base.file.FileException: ObjectFile