On Fri, Aug 12, 2016 at 1:31 PM, Andrew Stitcher <[email protected]> wrote:
> I should state here that I've not actually ever implemented a store for > qpidd - those that have shold be at least lurking here and can identify > themselves! > Hey, I'm just glad someone responded :) > > But I think I some architectural understanding (well recollection > really!) of how they fit. > > The first point is that there are actually 2 "stores" in qpidd and they > are not well separated: > > 1. There is a store for broker configuration - in the linear/legacy > stores this is a completely separate sleepycat db. I'm not sure how the > windows stores separate. > > 2. A store for durable messages. > > I suspect you've already noticed this if you've got as far as > implementing a store. > > On Thu, 2016-08-11 at 17:43 -0400, Matt Broadstone wrote: > > On Thu, Aug 11, 2016 at 10:42 AM, Matt Broadstone <[email protected] > > > > > wrote: > > > > > > > > Hey all, > > > > > > I've been working for the past week on store plugin for RethinkDB > > > for > > > qpidd, and while I have made some pretty good progress I have a few > > > outstanding questions I was hoping someone here might have insight > > > into. > > > Here goes: > > Are you planning to open source this plugin? It would be great to have > another store that worked on Unix, but wasn't tied to Linux only APIs. > > Yes I am, in fact I had briefly discussed with Gordon the possibility of letting the source live in-tree. Presently the plugin seems to work quite well, with a notable exception of transaction support. RethinkDB doesn't actually support transactions, but I imagine I could come up with something here if it was a requirement. > > > > > > There seem to be some notable differences between the > > > linearstore/legacystore and the MSSQL plugins. > > As you say they conform to different interfaces. > > As I understand it that is because the linear/legacy stores were > written in a way that assumes that only one store can be module loaded. > The legacy store was the first store written for the current store > interface, and it was simplest to just assume that only one store could > be loaded > > Whereas the Windows stores (I believe that only uses MSSQL and the > other uses a different native journalling API) can both be loaded, but > opnly one can be active. These were written afterwards and I'm guessing > that because there were two written at the sam e time an indiection > layer was also written to switch between them. > > It would make sense to switch the linearstore to use the layered API so > that multiple stores could be loaded and switched, but there has been > no interest in doing that work, largely because there has been no other > Linux store. > > Yes, this was my understanding as well. I've been targeting making this a "StorageProvider" ala the MSSQL drivers. > > > If I'm not mistaken (and > > > please correct me if I'm wrong here), the MSSQL actually seems to > > > be more > > > of a reference implementation of the plugin api than the first two, > > > in that > > > the builtin stores are `qpid::broker::MessageStore`s while the > > > latter is a > > > `qpid::store::StorageProvider`. Which one should I be targeting > > > here? > > If I remember correctly StorageProvider is the interface which allows > multiple stores to be loaded at once (although only one can be active > at any one time). > > Architecturally I think StorageProvider sits on top of MessageStore. > > > > ... {Details that I know nothing about) > > I was really hoping someone could speak to these methods :) Perhaps someone else will chime in here. > > > Finally I just have some more general questions: > > > - are there any resources available for implementors that I might > > > have > > > missed? Perhaps design documents, or some treasure trove of > > > communications > > Sadly I'm pretty sure there is nothing available. The legacystore used > to be maintained as a separate plugin in a different repo, if you can > find that there may be something there. > > > > ... > > > Just another quick observation as I was cleaning up my cmake rules. > > Currently the "store" library is built as a module which I believe is > > incorrect but has probably worked so far since the only consumer has > > been a > > windows build. > > Is the "store" plugin the one that allows switching between different > stores? If so then I agree it should be built directly into qpidd. > > Yes the store "plugin" is really just a shared base for storage providers it seems. It provides common definitions of the interfaces that need to be implemented by the store plugins. > However this can only happen if the linear/legacy stores are modified > to use the API it exposes, as otherwise there would be two store > plugins vying for the exclusive right to be a store using that plugin > API. > > I actually don't think this is correct. I believe the linear/legacy stores don't use the store plugin (or its interfaces) at all. > > Naturally, the store library should be a shared library that > > is directly linked into the storage providers that depend on it (you > > wouldn't `--load-module store --load-module mssql_store` just > > `--load-module mssql_store`). It seems on windows that module's are > > defined > > the same as shared libraries, so this probably worked just fine, but > > on > > linux it's a no-go and so linux store modules need to compile the > > parts of > > store that they need. > > To be clear, on Linux the "store" module isn't used or needed because > the current Linux stores don't use it! > > I hope this random spew of thoughts has been helpful or at least > amusing. > > Andrew > > Thanks for helping the cause here, I'm looking forward to any other information I can squeeze out of the list! > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
