Are you both (and anyone else interested) going to work on a draft blueprint before the summit? I'm keen to see discussion drafts in place before we all meet in person, otherwise we'll waste a lot of time just getting on the same page.
Thanks, Ewan. On Mon, Oct 04, 2010 at 07:01:57PM +0100, Eric Day wrote: > ++ on all points. When it was decided to implement a Redis backend for > the DB API, I tried to explain my concerns about recreating SQLAlchemy > with a Redis back-end, and then some. We should decide what data > model we are using (relational, hierarchical, flat key-value, graph, > ...) and decide on a native API to use. If we wan to stay relational, > using SQLAlchemy natively and using it's pluggable backend system > may be sufficient. > > There is also the distributed storage and privacy concerns to discuss > as well at the summit. For example, each compute/network/volume worker > could keep it's own store and push any required information up to > the other layers (scheduler/API). This would ensure a compromised > worker machine could not touch other workers or their data stores, > and scaling a central data store is no longer an issue. With these > concerns in mind, this may also sway which data model and API we want > to expose to the applications. > > -Eric > > On Mon, Oct 04, 2010 at 01:34:27PM -0400, Jay Pipes wrote: > > I agree with Soren that big changes like this should occur after some > > discussion (a thought that I rose on IRC last week). > > > > I look forward to the discussion at the next summit about the ORM in > > general, and specifically, these concerns about it I have: > > > > 1) Inconsistent (and sometimes incorrect) use of session boundaries in > > the abstraction layer for sqlalchemy. Peculiar bugs show up when > > sessions aren't handled (passed between functions) properly or when > > transactions are not reliable (NoSQL or when the RDBMS doesn't support > > transactional statements like SELECT ... FOR UPDATE. > > 2) Shared code between Glance and Nova (and possibly Swift, but it's > > less of an issue IMHO). There is now a copy of the datastore adapter > > from Nova in Glance (this was done to save time). We will want to > > pull this code out into openstack-common ASAP, as I really don't feel > > like maintaining two copies of the datastore adapter layer ;) > > 3) The Model class itself. The biggest issue is that the model class > > does not live in the API layer; instead, it must be re-implemented for > > each adapter. I experienced the joy of trying to re-create the > > existing SQLAlchemy-based NovaBase model class in the Redis adapter. > > This was less than enjoyable. If we choose to use a base model class, > > it needs to address these fundamental issues *at a bare minimum*: > > a) The definition of data fields (versus non-data fields, private > > fields, or calculated fields) > > b) The casting of SQL-like data types (CHAR, INT, BLOB, DATETIME, > > etc) into data type that Python understands consistently. > > c) A model's understanding (or lack of understanding) of its > > relationships with other types of models. > > d) The aforementioned debate about what is appropriate as a primary > > key identifier for each model (UUID, URI, auto-inc, etc) > > > > -jay > > > > On Sun, Oct 3, 2010 at 5:10 PM, Devin Carlen <[email protected]> wrote: > > > Soren, > > > > > > This comment was with regards to using declarative_base(), which was used > > > to quickly implement the data layer and save some time. The next > > > iteration we've been discussing internally, is to drop the use of > > > declarative_base and using sqlalchemy table definitions instead. > > > > > > declarative_base() provides value primarily when you are planning on > > > actually using sqlalchemy models as the middle tier. We only use them as > > > a stepping stone in the data layer, and then treat everything as dicts > > > currently. (Whether we should be using dicts everywhere is part of a > > > larger discussion I hope to bring up at the next design summit.) > > > > > > By using the table definitions directly and bypassing declarative_base(), > > > we will be able to cut out two superfluous and unnecessary > > > transformations before the data is passed back to the middle tier. > > > > > > There are no immediate plans to do this, but you are right - it is worthy > > > of further discussion. > > > > > > > > > Devin > > > > > > > > > This only was meant to indicate that we would be changing from u > > > > > > On Oct 2, 2010, at 2:52 AM, Soren Hansen wrote: > > > > > >> Looking at the commit message for r316 on trunk, I see this (emphasis > > >> mine): > > >> > > >> Also refactored NovaBase and removed several methods since they > > >> would have to be changed *when we move away from sqlalchemy models* > > >> and begin using sqlalchemy table definitions. > > >> > > >> I find it worrisome that such things are stated as though they are > > >> naturally given or generally agreed upon. I have no recollection of a > > >> discussion to that effect, let alone a decision. > > >> > > >> I certainly hope a big patch to do this doesn't suddenly turn up without > > >> prior discussion and a related blueprint.. > > >> > > >> -- > > >> Soren Hansen > > >> Ubuntu Developer http://www.ubuntu.com/ > > >> OpenStack Developer http://www.openstack.org/ > > >> > > >> _______________________________________________ > > >> Mailing list: https://launchpad.net/~nova > > >> Post to : [email protected] > > >> Unsubscribe : https://launchpad.net/~nova > > >> More help : https://help.launchpad.net/ListHelp > > > > > > > > > _______________________________________________ > > > Mailing list: https://launchpad.net/~nova > > > Post to : [email protected] > > > Unsubscribe : https://launchpad.net/~nova > > > More help : https://help.launchpad.net/ListHelp > > > > > > > _______________________________________________ > > Mailing list: https://launchpad.net/~nova > > Post to : [email protected] > > Unsubscribe : https://launchpad.net/~nova > > More help : https://help.launchpad.net/ListHelp > > _______________________________________________ > Mailing list: https://launchpad.net/~nova > Post to : [email protected] > Unsubscribe : https://launchpad.net/~nova > More help : https://help.launchpad.net/ListHelp _______________________________________________ Mailing list: https://launchpad.net/~nova Post to : [email protected] Unsubscribe : https://launchpad.net/~nova More help : https://help.launchpad.net/ListHelp
