Hi Aron and all
I have read your proposal with great interest. As we could establish on
the committers meetings, I have been thinking along the same lines. You
have been doing most of the work for me, which makes me glad. I hope you
too will be glad, when I piggyback what you started.
You correctly highlight one shortcoming of Fedora, but fail to highlight
another. Your excellent paper only concerned itself with storage, but
caching is equally important.
Here is my counter proposal:
Fedora has a number of storage systems. Each of these are represented by
a class implementing an interface like this
public interface WritableStore {
void add(DigitalObject object);
void update(DigitalObject oldVers,
DigitalObject newVers);
void remove(DigitalObject object);
}
I am sure you can see where I got my inspiration ;) I used your version
(delete takes an object, rather than a pid for no particular reason).
But I removed the read() method. All fedora stores should be able to
digest fedora objects. But not all of them should be able to put an
object back together again.
Fedora maintains two lists of such stores, one for synchronous updates,
and one for asynch updates.
Whenever fedora receives a request to change something, it sends the
request to each of the synchronous stores, queue it for the asynchronous
stores and return.
It will have one ReadableStore with a interface
interface ReadableStore {
DigitalObject read(String pid);
}
Whenever objects need to be read, this is the store that is read from.
Interestingly, this does not need to be any one of the WritableStores.
You could have a number of Writable stores doing journaling of each
other, and having the WritableStore implementation alternating between
them when objects should be read.
Fedora, in the current incarnation, uses these stores
Synchronous
* Lowlevelstore
* Database
* Messaging
* Gsearch (by DOManager)
* Lodewijks custom database integration
Asynchronous
* Triple store
* Search engines (by messaging)
By making all these stores adhere to at standard interface, we could
easily change which are synchronous and which are not from the config
file. We could easily add new stores and caches and indexes. Store
integration could be shared as binary files, with should just be
installed in fedora, and declared in fedora.fcfg.
We could have multiple redundant stores, and having the ReadableStore
implementation alternate between them.
With this approach, Fedora becomes
* A webservice interface
* An object model
* A system for object access policies
* Glue for binding stores together
I think I imagine these stores to live alongside the HighlevelStorage
system, in that the HighlevelStorage is just one implementation. But
they might also live below the HighlevelStorage.
So, what do you think?
Regards
Asgger
On Wed, 2010-01-27 at 07:12 +0100, Aaron Birkland wrote:
> Hi all,
>
> As you may be aware, there is a face-to-face meeting in London where
> various committers and (other interested parties) will be discussing
> future directions for Fedora. One topic slated for discussion is that
> of Fedora's storage architecture. The ideas of multiplexing storage (so
> that managed content can be sent to different storage devices based on
> some criteria), hierarchical storage, alternate object representations,
> etc have been discussed in the past. I would like to propose a few
> architectural changes to the Fedora core that could make these tasks
> easier to implement.
>
> I made a skeleton of a page on the wiki:
> http://fedora-commons.org/confluence/x/syTS
>
> A draft is currently written up in a pdf doc (which is linked on the
> wiki page):
> http://www.fedora-commons.org/confluence/download/attachments/13771955/fedora-highlevel-storage_draft-proposal.pdf
>
> The main purpose of the proposal is to spur conversation on the topic..
> the proposal may or may not represent an ideal approach, but it seems to
> represent a logical next step that would remove some of the existing
> architectural barriers to implementing more sophisticated storage
> strategies.
>
> I decided to put this up quickly so that people can have some time to
> read it over, ask questions, discuss, think about how to improve it, or
> propose alternatives. Hopefully, this is a topic that people would like
> to discuss.
>
> -Aaron
>
>
> ------------------------------------------------------------------------------
> The Planet: dedicated and managed hosting, cloud storage, colocation
> Stay online with enterprise data centers and the best network in the business
> Choose flexible plans and management services without long-term contracts
> Personal 24x7 support from experience hosting pros just a phone call away.
> http://p.sf.net/sfu/theplanet-com
> _______________________________________________
> Fedora-commons-developers mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers
------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
_______________________________________________
Fedora-commons-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers
