Re: javax.jnlp.PersistenceService discoveries and questions about Derby StorageFactory/StorageFile - help me out pls :)

[Luigi Lauro]

On 23 Mar 2007, at 15:37, Daniel John Debrunner wrote:

I think removal of the temp files happens above the Storagefactory,

Just looked at the code and that comment already exists :-)
  StorageFactory.getTempDir().

but it would also be ok for the Storagefactory to remove them on shutdown. Adding comments to the javadoc for StorageFactory would help here, e.g. temp files need not survive shutdown.

Ok, so if I get it right, I can implement temporary files in a non- persistent way, but, even if they are made persistent, they are deleted by higher-love code above StorageFactory so I don't have to deal with deleting them. Good to know.

B) StorageFactory.init() - Let me see If i Got this right.

DatabaseName is subdir in home for the given database.

UniqueName is database specific subdirectory inside tempDirName.

Home is directory of derby home (where all database are stored), but can be ignored. TempDirName is home for temp files (can temporary files be created also outside of it, like in database directory? Is this possible?) Home shouldn't be created but provided (though in my case I have to create it since the user can't). DatabaseName can be null when you are using the StorageFactory just to access the database directories (this will happen with my factory as well?), but if its not I have to create home/databaseName. TempDirName can be null and a default should be used, and the directory created, but ONLY if uniqueName is not null. If uniqueName is null, then no temp dir is available and from what I can guess it means Derby won't use ANY TEMPORARY FILE AT ALL. Right?

Derby will create temp files in the database location. tempDirName (I think) is for when the database StorageFactory is read-only (e.g. database in a jar file) thus temp files need to be elsewhere.

Ok, so usually temp files are created inside the database directory itself, but in case that is read-only, a temp dir for writable temporary files is needed, and that's what tempDirName/uniquename serves for. Ok, fine, now I start to get it, wasn't completely clear reading the javadocs alone.

Also, please use better names next time. IMHO home, databaseHome, temp, databaseTemp would have been a wiser choice, more easily understood by non-derbiers. Anyway.

derby.system.home.

home is not specific to a single database, thus databaseHome may not be the correct term, it's really systemHome, as it comes from the property Same for temp, it's a top-level folder that can contain temp files for many databases.

No, i meant for example

/derby (home)
/derby/database (databaseHome)
/derbytmp (temp)
/derbytmp/uniquename (databaseTemp)

These names IMHO make things clearer: make you understand home is home, databaseHome is home just for the given database, and temp is temp home, databaseTemp is specific temp for the given DB. They are more easily understood than home/database/tempDirName/uniqueName, which aren't straight clear unless you read the javadocs :)

Feel free to submit a patch with better names for parameters (that won't change the api) and/or improved comments.

Didn't mean to flame here, sorry for the harsh comment. I just meant to give my POV of a non derbier approaching StorageFactory interface / javadocs, and what IMHO can be improved :)

C) StorageFactory.newStorageFile(String path) - should this method also create a temporary file, if the given path is under the temp dir? This isn't clear IMHO: if they hand me a tempDir path, do I have to just wrap it in a StorageFile, or Do I also have to create a temporary file with a unique name and return it? I Really haven't clear all the newStorageFile/createTemporaryFile methods, and even reading the javadocs and looking at BaseStorageFactory, I still feel puzzled.

Can someone help me understand this? I'd love that :)

I'll try to find time to add some understanding ...

In this environment can you use File.createTempFile()?

Yes, I could probably delegate the non persistent temporary files to a real temp file, this is a possible solution. Another one would be to implement them as in-memory temp files, which would probably be even better.

D) This is hard to explain, but I will try my best. Since the PersistenceService API only gives me a 'name' metadata for a given storage entity, and I prefer not to save metadata inside the file contents itself (this would make things harder to implement), I was asking myself what metadata regarding the file do I have to keep. Surely, first of all, I would need to tell if a storage entity it's temporary or not. If temp files gets created ONLY under the tempDir, this means I could use the name/path itself to tell this. But I'm still wondering if derby creates temporary files also outside tempDir or not. I think and hope not. Also, I surely have to tell if a given storage entity it's a directory or a file. I could tell them by the name/path as well (if they end with SEPARATOR, they are dirs, if they don't, they are files), but this would work ONLY if derby doesn't craft itself alone paths for directories/files by parsing the path instead of using the StorageFactory/StorageFile methods. If derby does, then derby could produce a directory URL without the trailing separator, which will mess things up. Another approach would be to use the use a zero maximum length as directory tag. A directory would have a zero length (and this is good also, since I'll create directories storage entities just as a placeholder, to tell if a directory was created or not), a file would have a default 1024/whatever maximum length at start (remember that this can be easily grow when needed, when I write to the entity). Again, there is also a readonly bit on the files. Do I have to persist this as well? It would be a problem if I 'lose' this information and don't persist this? Also: are there any other file metadata bits which I'm forgetting and that I should save in the persistent storage?

I think that is set only by the storage factory, i.e. it is telling derby if this is a read-only database or not.

Good to know, then no need to persist it.

Thanks again for your help, If I get some answers to shed some lights onto my doubts, I will fix a couple of things in the next few hours and post an 'alpha' JNLPStorageFactory/StorageFile patch in the jira issue. Also, since working on it I've found out there are SEVERAL similarities between JNLP storage and an hypothetical memory storage using a simple Map<Path, File>, I'm implementing things around an abstract base class which delegates to the extending implementation classes only a couple of CRUD methods (create/ delete/rename/etc...) and build all the StorageFactory/StorageFile logic on top of these. This will mean I can probably get done a working MemoryStorageFactory along with the JNLP one, since doing the former would be only a 5% work more than doing my own JNLP storage, as I'm currently planning to do things.

Cool.

Also, I think this could potentially lead to a massive StorageFactory/StorageFile redesign for easier storage implementations, or to some higher-level abstract class wrappers around the present StorageFactory/StorageFile interfaces, such as the one I'm doing, that depends only on a FEW storage methods, instead of the 30+ methods one presently needs to implement to get a Storage working. Of course, this is something I can't really tell if this is something needed and good for real or not, given my very basic derby internal knowledge.

Sounds interesting.

Dan.

Thanks Dan. :)

smime.p7s
Description: S/MIME cryptographic signature