On 12/29/13 7:43 AM, "Ola Fosheim Grøstad"
<ola.fosheim.grostad+dl...@gmail.com>" wrote:
On Sunday, 29 December 2013 at 15:05:31 UTC, Andrei Alexandrescu wrote:
Well one question is what other successful designs could we use as
precedent? I don't know of any successful unified APIs for
regular/remote filesystems that also allow full local file
functionality. The closest abstraction I know of is the FUSE-installed
filesystems, but those adapt foreign file systems to the Posix interface.
I guess file storage design is currently in flux and a bit lacking in
terms of APIs, partially because of cloud computing and SSDs, so my
suggestion would be to:
1. Provide os.linux, os.windows, os.osx, os.ios, os.posix for low level
local-storage access which closely resembles the native/standard APIs.
I think we (and others) have done a fine job so far at abstracting away
e.g. very different ways of figuring whether an entry is a directory on
Windows vs. Posix. Now we're to throw all that away and go back to silex
stones and bear claws?
2. Provide a novel hierarchy of abstraction for basic file I/O that is
highly portable and afford modular and extensible implementation, and
make those really easy to use in order to encourage portable libraries
for D.
Yah, it would be great if that were awfully more detailed :o).
I don't think the overhead of function calls through interfaces
matters all that much because the cost associated with context shifts
for system calls will dominate that by a solid margin (my assumption).
That's exactly what they said when they designed iostreams.
So basically on the most abstract and limited level you get a key/value
cache (memcache). The next level is key/value datastore
(couchdb/mongodb/amazon/google etc).
Then the ability to list all keys.
Then the ability list keys based on hiearchy (directories) etc.
If you can specify what kind of functionality your application needs and
set those constraints in a file-system context object then you can set
the level of portability for your application. I think this would be a
real advantage. Then let that file-system object have a factory for
"file-mode" and let that do the right thing when you attempt to set the
filetype as a MIME-constant (e.g set the right extension or set the
right attribute).
Another advantage is that you could create "safe virtual filesystems"
e.g. create a file-system context object that resolve ".." and prefix
all paths with "/tmp/" and set default "file-mode" for created objects
(like read-only and owner). Which would make it easy to write more
secure web-servers. You could even specify a name-mangler and create a
db-like filesystem for arbitrary keys with automatic creation of
directories (for efficiency) if the abstraction level is right.
I don't think the overhead is all that much if it is done in a modular
fashion. You only import what you need. Finding the right abstraction is
not trivial, I agree. So you need to do some analysis of what current
storage-solutions provide to find the common denominators.
(Having done some work with ndb on App Engine and the HttpRequest stuff
in Dart I see the advantage of wrapping up resource loading as Futures.
Mostly sugar, but uniform and easy to remember if used on all resources.
E.g. request all resources as futures, then process them as they become
available.)
Again I think we'd need a ton of detail here to even assess whether this
has merit. So far it's an interesting brain dump.
Andrei
