Hi Barry,
Whatever you do, the overall distributed system has to deal with sudden
shutdown of a server. Because it happens.
One thing I've been wanting to do is dynamic restart - reread the
configuration file while the server is running and gracefully switch
over to the new configuration. Having a formal service lifecycle would
be on the path to that.
How are you finding the dataset?
If you haven't found it, at the moment, you can find all the
dataservices by walking the server setup:
FusekiServer server = ...;
server.getDataAccessPointRegistry().forEach((name,svc)->{
svc.getDataService().shutdown();
});
Not beautiful, but, weakly, principled.
This probably works after FusekiServer.stop has been called. The Fuseki
server itself is still there untouched (current design), it is just that
Jetty has stopped. There is going to be a timing issue if done before
JettyServer.stop.
Seem to be what is needed is FusekiServer shutdown hooks, one of which
is DataService.shutdown() coupled with a grace period. Any mechanism
needs to protect itself from faulty shutdowns that block for a long time.
This is Fuseki-main which is Jetty specific and Jetty has mechanism for
shutdown; I haven't investigated. Fuseki-full is a webapp for any
webapp container server; it could tap into the end of the ServletContext
in FusekiServerListener.
Caution: as it is today, DataService.shutdown does a TDB expel if it can
find a TDB dataset in nested datasets (e.g. text index). That throws
the TDB DatasetGraph out of the system without ceremony.
It is not thread safe. No transaction can be running at the time.
It does not loose data. All caching is lost.
It is really for testing.
You can not reconnect to the database if running on MS Windows (long
time Java bug).
Andy
On 27/03/2020 07:52, Nouwt, B. (Barry) wrote:
Hi Andy, thanks for your reply! Interesting read, that's some impressive
digging indeed.
I was wrong on the DatasetGraph.close() call, the FusekiServer.stop() method
just calls JettyServer.stop(). I assumed it would close the DatasetGraph via
the DataAccessPoint and the DataService.shutdown(), but it doesn't call
DataService.shutdown() anywhere.
Our custom dataset has registered itself to some external service and it would
be nice if it would unregister itself before shutdown, but you are right that
we should not rely on that and take measures on 'the other side' as well. The
custom dataset also holds an Apache ActiveMQ instance with several
subscriptions and a ExecutionService that we would like to clean up.
What would be your advice? Should we not even try to shutdown gracefully using
the shutdownHook and just take measures on the other side to detect the custom
dataset is no longer available?
Thanks again! Regards, Barry
-----Original Message-----
From: Andy Seaborne <a...@apache.org>
Sent: donderdag 26 maart 2020 17:12
To: users@jena.apache.org
Subject: Re: Graceful shutdown of FusekiServer
Because servers have to survive disgraceful shutdown anyway (crash, DOS, server switched
off, casters on the rack broken [1]), there isn't a graceful procedure. "kill
-9"
TDB has to be able to recover in any circumstances so that it what it does. No
graceful shutdown currently.
What resources of your custom dataset are there?
Andy
[1]
https://cloud.google.com/blog/products/management-tools/sre-keeps-digging-to-prevent-problems
On 26/03/2020 10:46, Nouwt, B. (Barry) wrote:
Hi everybody,
We are using embedded FusekiServer in our Java project and have extended Jena's
DatasetImpl class with additional features that require some cleanup when the
FusekiServer shuts down. This dataset is loaded into FusekiServer using a
config.ttl file, so we do not have a direct reference to this custom Dataset
object in our application.
To release these resources of our custom dataset, we implemented the
Dataset.close() function, but it is not being called when we shutdown the
FusekiServer using FusekiServer.stop().
Fuseki works in DatasetGraphs (DSGs). Dataset (usually) a wrapper to give a
different API.
As SPARQL goes down to the DSG dataset does do much for SPARQL.
I did find that the FusekiServer calls the DatasetGraph.close() function
instead, which is encapsulated by our KnowledgeDataset.
At what point does it call DatasetGraph.close()?
What is the best way to release the resources of our custom dataset
implementation? For now, I've added a shutdownhook to the JVM, but in my
opinion this is not very elegant.
Agreed. Even if there isn't an elegant way now (there maybe, there may not),
let's at least understand what it would take to add.
Thanks in advance!
Regards, Barry
This message may contain information that is not intended for you. If you are
not the addressee or if this message was sent to you by mistake, you are
requested to inform the sender and delete the message. TNO accepts no liability
for the content of this e-mail, for the manner in which you use it and for
damage of any kind resulting from the risks inherent to the electronic
transmission of messages.