Re: [sqlalchemy] Feedback appreciated
On Aug 23, 2013, at 3:52 PM, Konsta Vesterinen wrote: > > It would be great if some of these packages could become official SQLAlchemy > plugins/extensions some day. It would also be great if those extensions could > be magically registered as sqlalchemy extensions in Flask-esque way (eg. from > sqlalchemy.ext.continuum import VersioningManager). > > I'd like to see a section in SQLAlchemy website of 'official' extensions. If > you feel some extension is mature and good enough it could be put there to > gain good visibility. Currently we could put GeoAlchemy in there? well we've talked about SQLAlchemy itself supporting some kind of namespace package, but so far I haven't been enthused about that. For one, the namespace package thing seems to keep changing, so not sure I'd want a huge ecosystem based around a somewhat moving spec (it's not as big a deal if you control all the projects directly). For another, I get a strong impression that namespace packages make the external lib look like it's part of the library itself, especially to newbies, and the community of SQLAlchemy packages on pypi is really large and of very variable quality, I'd rather have any namespace that starts with "sqlalchemy" be something that can be centrally curated, rather than allowing anyone to name their package "sqlalchemy.contrib" - not to mention if some very low quality package takes over some very common name, like "sqlalchemy.contrib.versioning", then everyone downloads that, when meanwhile there are eight other great versioning systems that nobody finds out about because they don't have the cool namespace. then if the author of "versioning" just vanishes, now we have "sqlalchemy.contrib.versioning" is like a permanent deadweight. > > Sure! You could add this text: > > 'Fast Monkeys is a product development house based in Finland. We develop new > web ventures using Python, Flask and SQLAlchemy.' > I put you over here: http://www.sqlalchemy.org/organizations.html#fastmonkeys signature.asc Description: Message signed with OpenPGP using GPGMail
Re: [sqlalchemy] attribute_mapped_collection use as a key->list dictionary
On Sep 4, 2013, at 7:08 AM, Paul Balomiri wrote: > > My problem is related to the fact that the list values of the GBK defaultdict > are plain list types, and thus cannot fire events for operations on them. The > testcase below does not work, and, as you mentioned, no other operation on > GBK's list values will fire the corresponding events. > > Now my attempt (admittedly without enough knowledge of sqlalchemy internals) > was to create a list which *forwards* append/remove events > to the GBK Collection which could in turn add/remove them in their quality as > true InstrumentedAttribute (thus handling the DB part) . So more specifically > i used prepare_instrumentation() hoping to be able to instantiate an > InstrumentedList with event capabilities. The InstrumentedLists would not > need be first class InstrumentedAttributes ( -- perhaps could not because > they appear after reflection time? --). OK let's go back to my original reply, this is the key sentence: "In particular I stopped at getting Person.addresses_by_role['role'].append(Address()) to work, since that means we'd need two distinctly instrumented collections, it's doable but is more complex. " that is, when we work with the list collections that are in the dict, *that* part is not instrumented, it is a plain list. That's why the implementation has the methods "add(key, value)" and "remove(key, value)", which do the dictionary + list access in one step - direct mutation of the contained list is not part of any instrumented system and was omitted from my original example. For those to work, we need two new collections - there is no magic within the existing collection system that has any clue about these completely different kinds of collections, so the event system, prepare_instrumentation, etc, none of that applies here. The event system is not relevant here, as that system allows consumers to be associated with producers, but in this case, we're the producer! there is no shortcut and this is not anything built in to SQLAlchemy - these are totally new kinds of collections and all four interfaces (dict, list, proxied dict, proxied list) must be fully implemented, if you truly want completely transparent behavior. Here are all four, not completed yet, might have bugs, but implementing instrumented (and proxied) access on the sub-list element. from sqlalchemy import * from sqlalchemy.orm import * from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.orm.collections import collection, collection_adapter from sqlalchemy.ext.associationproxy import association_proxy, _AssociationCollection, _AssociationList Base = declarative_base() class GroupByKeyCollection(dict): def __init__(self, keyfunc): self.keyfunc = keyfunc def __missing__(self, key): l = GroupByKeyList(self, key) dict.__setitem__(self, key, l) return l @collection.appender def add(self, value, _sa_initiator=None): key = self.keyfunc(value) self[key].append(value, _sa_initiator) @collection.remover def remove(self, value, _sa_initiator=None): key = self.keyfunc(value) self[key].remove(value, _sa_initiator) @collection.internally_instrumented def __setitem__(self, key, value): if key in self: old_list = self[key] to_remove = set(list_).difference(value) for item in old_list: if item in to_remove: old_list._remove_event(item) else: old_list = [] to_add = set(value).difference(old_list) new_list = self.__missing__(key) for item in value: if item in to_add: new_list._append_event(item) @collection.internally_instrumented def __delitem__(self, key): if key in self: existing = self[key] for value in existing: existing._remove_event(value) dict.__delitem__(self, key) else: raise KeyError(key) @collection.iterator def iterate(self): for collection in self.values(): for item in collection: yield item @collection.converter def _convert(self, target): for collection in target.values(): for item in collection: yield item def update(self, k): raise NotImplementedError() class GroupByKeyList(list): def __init__(self, parent, key): self.parent = parent self.key = key def append(self, value, _sa_initiator=None): value = self._append_event(value, _sa_initiator) list.append(self, value) def remove(self, value, _sa_initiator=None): self._remove_event(value, _sa_initiator) list.remove(self, value) def _append_event(self, value, _sa_initiator=None): adapter = collection_adapter(self.parent) return adapter.fire_append_event(
Re: [sqlalchemy] query question
On Wed, Sep 4, 2013 at 12:05 PM, lars van gemerden wrote: > I think i must be reading over something, but: > > is there a way to delay the selection of attributes in a query; something > like > > >>> session.query(Person).filter(Person.age > > 100).select(Person.name).first() > > >>> (u"Ancient Bob",) > I'm not sure quite what you mean, but the with_entities method of Query would seem to be the closest to your example: http://docs.sqlalchemy.org/en/rel_0_8/orm/query.html#sqlalchemy.orm.query.Query.with_entities I assume you know you can also do this: session.query(Person.name).filter(Person.age > 100).first() You can also defer certain columns so that they will be loaded lazily: http://docs.sqlalchemy.org/en/rel_0_8/orm/mapper_config.html#deferred-column-loading Hope that helps, Simon -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] query question
Haven't tried it myself, but I think you can use the add_columns method of Query. Check out http://docs.sqlalchemy.org/ru/latest/orm/query.html On 4 Sep, 2013 2:05 PM, "lars van gemerden" wrote: > I think i must be reading over something, but: > > is there a way to delay the selection of attributes in a query; something > like > > >>> session.query(Person).filter(Person.age > > 100).select(Person.name).first() > > >>> (u"Ancient Bob",) > > Cheers, Lars > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy+unsubscr...@googlegroups.com. > To post to this group, send email to sqlalchemy@googlegroups.com. > Visit this group at http://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] attribute_mapped_collection use as a key->list dictionary
Hi,
First off and again, thanks for the support here. I think I can work for
the momement with your proposed implementation.
My problem is related to the fact that the list values of the GBK
defaultdict are plain list types, and thus cannot fire events for
operations on them. The testcase below does not work, and, as you
mentioned, no other operation on GBK's list values will fire the
corresponding events.
Now my attempt (admittedly without enough knowledge of sqlalchemy
internals) was to create a list which *forwards* append/remove events
to the GBK Collection which could in turn add/remove them in their quality
as true InstrumentedAttribute (thus handling the DB part) . So more
specifically i used prepare_instrumentation() hoping to be able to
instantiate an InstrumentedList with event capabilities. The
InstrumentedLists would not need be first class InstrumentedAttributes ( --
perhaps could not because they appear after reflection time? --).
I see now that it was a far off longshot.
This approach would also remove the immutability constraint on keyfunc's
return value. Keyfunc would be a read/writable attribute instead.
I hope to find time to get more accustomed to sqlalchemy's internals and to
implement this idea somewhere in Jan. or Feb..
By the way I'm implementing Rails-Style DB binding (table people->Class
Person e.t.c.) using the SQLA inspector interface. I'll post the lib in
github as soon as I'm eating my own dog food (=it's usable) .
--
Paul
Here is a testcase where i would like to have the list
p1._addresses_by_role["home"] fire a remove event which removes the person
from GBK:
def test_del_item(self):
sess = self.sess
p1 = Person()
a1 = Address(name="Bucharest")
# here, p2a already refers to a1/p1, the "_addresses_by_role"
# will be set up when it loads after a commit
p2a = PersonToAddress(address=a1, person=p1, role="home")
sess.add(p1)
sess.commit()
self._assertAssociated(p1, a1, p2a)
del p1._addresses_by_role["home"][0]
sess.commit()
import pdb
pdb.set_trace()
self.assertFalse("home" in p1._addresses_by_role)
2013/9/4 Michael Bayer
>
> On Sep 3, 2013, at 8:47 AM, Paul Balomiri wrote:
>
> > I would like to install
> >
> > event.listen(list, 'append', append_listener)
> > event.listen(list, 'remove', rm_listener)
> >
> > on those lists, such that the GroupByKeyCollection can modify added
> objects according to the relationship it implements:
> > * set the appropiate foreign key constraints
> > * insert a removed object with it's new value for the key attribute
> after a change (announced by append_listener)
> > * reset the fks upon item removal.
>
> using event.listen with GBK doesn't make sense. events can only be used
> with specific target types, the "remove" "append" events only apply to an
> ORM-produced InstrumentedAttribute, such as Person._addresses_by_role here
> (note, we mean the class-bound attribute, not the collection on an
> instance). There is no need to use event.listen with the collection
> itself, as remove/append are produced originally by the add()/remove()
> methods on GBK itself; any extra logic which should take place would be
> invoked directly from there (and in fact my original example fails to fire
> off the event with remove()).
>
> Additionally, all the usage of prepare_instrumentation() etc. should not
> be necessary, that's all internal stuff which is called automatically.
>
> As mentioned before, the behavior of this collection is completely outside
> the realm of a "normal" collection so it needs to implement the
> append/remove events directly, which isn't something a new user to
> SQLAlchemy would typically be able to handle without a much deeper
> understanding of how the attribute system works.
>
> I've implemented your test case as below as well as some other variants in
> association with the original code I gave you - for the "remove" case I've
> added the necessary code to the custom collection. All foreign key
> constraints are set correctly as a function of the ORM's normal operation,
> and as far as "reset", when an association between Person and Address is
> removed, we want to just delete the association so cascade is used for
> that. I'm not sure what "insert a removed object with it's new value for
> the key attribute after a change" means; add a test to the TestPA class
> illustrating the behavior you want and I'll add it.
>
> from sqlalchemy import *
> from sqlalchemy.orm import *
> from sqlalchemy.ext.declarative import declarative_base
> import collections
> from sqlalchemy.orm.collections import collection, collection_adapter
> from sqlalchemy.ext.associationproxy import association_proxy,
> _AssociationCollection
> Base = declarative_base()
>
> class GroupByKeyCollection(collections.defaultdict):
> def __init__(self, keyfunc):
> super(GroupByKeyCollection, self).__init__(list)
> sel
[sqlalchemy] query question
I think i must be reading over something, but: is there a way to delay the selection of attributes in a query; something like >>> session.query(Person).filter(Person.age > 100).select(Person.name).first() >>> (u"Ancient Bob",) Cheers, Lars -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
