[sqlalchemy] Re: SQLAlchemy 0.6.6 Released
So the C extensions would be good for someone that would use it for an ETL with a lot of data? :) On 9 jan., 17:25, Michael Bayer mike...@zzzcomputing.com wrote: I remain a little nervous about the C extensions, not as much because the current code is unreliable, but because I'd like there to be a whole lot more C code here. I've used the existing C extensions quite a bit and done tons of profiling - they really don't account for the vast majority of time spent in large ORM operations, and they don't do much for a non-ORM app that emits a lot of statements. They just cut down on an app that is non-ORM and needs to fetch very large result sets. Basically I'd need a resource that can do lots more C code for me, if not myself since it just would take a huge block of extra time for me to get into it, before I'd feel good about the C code overall and confident that I can put out releases in a timely manner which would now require 100% reliable C code. The limitations of distutils are also troubling here. In my own work app, we use pip in conjunction with a Makefile and for the SQLAlchemy install the flag is on in the Makefile. In that regard the flag being off by default doesn't feel like that big a deal to me personally. On Jan 9, 2011, at 3:33 AM, Wichert Akkerman wrote: Hi Mike, On 1/9/11 00:14 , Michael Bayer wrote: The majority of my time is now spent developing 0.7, which is nearly ready for beta releases pending a few more little features I'd like to try to get in. 0.7 is really exciting with its new event API, lots of other nice touches and of course the most radical reduction in callcounts we've had in a few years. Has been there a decision on enabling the C extensions by default in 0.7, or is that still too controversial? Regards, Wichert. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Insert record error
On Thursday, January 13, 2011, William Hudspeth bhudsp...@edac.unm.edu wrote: Thanks for responding, I tried changing the geometry object definition, as well as the model definition and I get the same result... dream_geom=MULTIPOLYGON((-120.000 43.833,-96.833 43.833,-96.833 26.000,-120.000 26.000,-120.000 43.833)) Your multipolygon WKT is ill-formed. Try with three opening and closing brackets. If that still doesn't work I suggest that we continue the discussion on the GeoAlchemy list. Cheers, -- Eric Lemoine Camptocamp France SAS Savoie Technolac, BP 352 73377 Le Bourget du Lac, Cedex Tel : 00 33 4 79 44 44 96 Mail : eric.lemo...@camptocamp.com http://www.camptocamp.com -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] unittest assertRaises does not catch NoResultFound
On 11/01/2011 15:26, Michael Bayer wrote: On Jan 11, 2011, at 4:56 AM, pr64 wrote: self.assertEqual(self.session.query(User).filter(User.firstname == user1_fn).count(), 1) self.assertRaises(NoResultFound, self.session.query(User).filter(User.firstname == other_fn).one()) assertRaises receives a callable, which when called raises the expected error. So you need to not actually invoke one(): self.assertRaises(NoResultFound, self.session.query(User). filter(User.firstname == other_fn). one ) Yeah, I find this pattern a bit clunky. Nowadays, context managers provide a much nicer way of doing these kinds of tests. Most testing packages have some flavour of this, my testfixtures package has ShouldRaise: http://packages.python.org/testfixtures/exceptions.html#the-shouldraise-context-manager cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: guard all session.query with try except?
On 14/01/2011 06:59, can xiang wrote: Thanks for your advise. My application is a tornadoweb app. So I'm going to create Session() for each request. Tornado is an async framework, right? If so, you may be in for a bumpy ride unless you get all SQLAlchemy stuff happening in threads... cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: SQLAlchemy 0.6.6 Released
An ETL (an acronym I keep forgetting, then I look it up, then I say, oh right I write three of those per week) is heavy on the flush side, assuming the E part is from spreadsheets or something like that.I made a few fairly vast improvements to flush() efficiency in 0.7, they're fun to watch in the logs. On Jan 14, 2011, at 4:28 AM, dusans wrote: So the C extensions would be good for someone that would use it for an ETL with a lot of data? :) On 9 jan., 17:25, Michael Bayer mike...@zzzcomputing.com wrote: I remain a little nervous about the C extensions, not as much because the current code is unreliable, but because I'd like there to be a whole lot more C code here. I've used the existing C extensions quite a bit and done tons of profiling - they really don't account for the vast majority of time spent in large ORM operations, and they don't do much for a non-ORM app that emits a lot of statements. They just cut down on an app that is non-ORM and needs to fetch very large result sets. Basically I'd need a resource that can do lots more C code for me, if not myself since it just would take a huge block of extra time for me to get into it, before I'd feel good about the C code overall and confident that I can put out releases in a timely manner which would now require 100% reliable C code. The limitations of distutils are also troubling here. In my own work app, we use pip in conjunction with a Makefile and for the SQLAlchemy install the flag is on in the Makefile. In that regard the flag being off by default doesn't feel like that big a deal to me personally. On Jan 9, 2011, at 3:33 AM, Wichert Akkerman wrote: Hi Mike, On 1/9/11 00:14 , Michael Bayer wrote: The majority of my time is now spent developing 0.7, which is nearly ready for beta releases pending a few more little features I'd like to try to get in. 0.7 is really exciting with its new event API, lots of other nice touches and of course the most radical reduction in callcounts we've had in a few years. Has been there a decision on enabling the C extensions by default in 0.7, or is that still too controversial? Regards, Wichert. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Insert record error
Eric, I have posted to the GeoAlchemy list after trying your suggestion... http://groups.google.com/group/geoalchemy/browse_thread/thread/5a0c2a2ddce2cfc9# Thanks On Fri, 2011-01-14 at 13:38 +0100, Eric Lemoine wrote: On Thursday, January 13, 2011, William Hudspeth bhudsp...@edac.unm.edu wrote: Thanks for responding, I tried changing the geometry object definition, as well as the model definition and I get the same result... dream_geom=MULTIPOLYGON((-120.000 43.833,-96.833 43.833,-96.833 26.000,-120.000 26.000,-120.000 43.833)) Your multipolygon WKT is ill-formed. Try with three opening and closing brackets. If that still doesn't work I suggest that we continue the discussion on the GeoAlchemy list. Cheers, -- Eric Lemoine Camptocamp France SAS Savoie Technolac, BP 352 73377 Le Bourget du Lac, Cedex Tel : 00 33 4 79 44 44 96 Mail : eric.lemo...@camptocamp.com http://www.camptocamp.com -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Best way to reduce boilerplate?
I'm having trouble getting mixins to play nicely with __table_args__.
Using the example I gave earlier in this thread:
On Jan 13, 1:00 pm, Michael Bayer mike...@zzzcomputing.com wrote:
On Jan 13, 2011, at 12:11 PM, Randall Nortman wrote:
[...]
namespace_id = Column(Integer, ForeignKey(Namespace.id),
nullable=False)
namespace = relationship(Namespace, backref='widgets')
name = Column(Unicode(256), nullable=False)
__table_args__ = (
UniqueConstraint(namespace_id, name),
{})
[...]
I can stick that code in a mixin class (converting everything but
'name' to be a @declared_attr), and it works just fine -- until a
class using the mixin wants to have some __table_args__ of its own.
The docs have an example of manually combining __table_args__ from
mixins in the class that uses the mixin, but I find that a little bit
ugly. In particular in this case, the mixin doesn't just use a dict
as its __table_args__, so the derived class needs to know that (or
test for that) and pull in the positional arguments and also the
dict. I don't like that, because the derived class needs to know too
much about the internals of the mixin, and I may change those
internals at some point, or else it needs some complicated code to
introspect the type of the mixin's __table_args__ (which may be a
sequence, a dict, or a @declared_attr method returning a sequence or a
dict).
Is there a nice solution that doesn't require a user of the mixin to
bend over backward to add its own __table_args__ without obliterating
the mixin's __table_args__? I would rather put complicated code in
the mixin rather than the user of the mixin, but if the user specifies
__table_args__, then the mixin's __table_args__ never seems to be
called in order to have the opportunity to do any magic.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Implementing a specific commit/refresh strategy for one table only
Thanks Michael. I'll also study carefully the other thread (can expire_on_commit be made...) it has interesting insights ! Franck On Thu, Jan 13, 2011 at 4:26 PM, Michael Bayer mike...@zzzcomputing.comwrote: You'd implement the expire yourself using SessionExtension.after_commit(). On Jan 13, 2011, at 9:18 AM, Franck wrote: Dear all, I've decided to plug SQLAlchemy to my web framework in order to let SQLAlchemy handle the framework's user sessions. Theses sessions require a lot of SELECT and UPDATE all the time. Therefore, I'd like to toggle expire_on_commit and (possibly) autocommit *for the SESSIONS table only.* The strategy for other tables should not change. Is it possible ? Here is how I define my scoped_session for the application : orm = scoped_session(sessionmaker(bind=engine)) Here's an example of use : now = datetime.datetime.now() s = Session.get(key) s.atime = now orm.commit() Thanks a lot ! Franck -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.comsqlalchemy%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Best way to reduce boilerplate?
On Jan 14, 2011, at 12:40 PM, Randall Nortman wrote:
I'm having trouble getting mixins to play nicely with __table_args__.
Using the example I gave earlier in this thread:
On Jan 13, 1:00 pm, Michael Bayer mike...@zzzcomputing.com wrote:
On Jan 13, 2011, at 12:11 PM, Randall Nortman wrote:
[...]
namespace_id = Column(Integer, ForeignKey(Namespace.id),
nullable=False)
namespace = relationship(Namespace, backref='widgets')
name = Column(Unicode(256), nullable=False)
__table_args__ = (
UniqueConstraint(namespace_id, name),
{})
[...]
I can stick that code in a mixin class (converting everything but
'name' to be a @declared_attr), and it works just fine -- until a
class using the mixin wants to have some __table_args__ of its own.
The docs have an example of manually combining __table_args__ from
mixins in the class that uses the mixin, but I find that a little bit
ugly. In particular in this case, the mixin doesn't just use a dict
as its __table_args__, so the derived class needs to know that (or
test for that) and pull in the positional arguments and also the
dict. I don't like that, because the derived class needs to know too
much about the internals of the mixin, and I may change those
internals at some point, or else it needs some complicated code to
introspect the type of the mixin's __table_args__ (which may be a
sequence, a dict, or a @declared_attr method returning a sequence or a
dict).
Is there a nice solution that doesn't require a user of the mixin to
bend over backward to add its own __table_args__ without obliterating
the mixin's __table_args__? I would rather put complicated code in
the mixin rather than the user of the mixin, but if the user specifies
__table_args__, then the mixin's __table_args__ never seems to be
called in order to have the opportunity to do any magic.
The testing of tuple/dict as well as combining several is left up to users at
this point. I'd suggest putting that logic into a library function so that it
isn't cluttering up individual __table_args__ methods. As far as user code
knowing about the mixin, that requirement is no different from user code that
uses an __init__() being aware the requirement of __init__() for superclasses.
It's currently an unsolved problem how conflicting tuples/dicts would be
reconciled automatically by declarative, and doing so would also make it
impossible for implementations to override its behavior, since declarative
would be forcing each individual __table_args__() to execute. So in this area
we kept the behavior the same as that which Python applies to any other method
- favoring simplicity of implementation over magic, within a process that
already is a little heavy on the magic side.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Best way to reduce boilerplate?
On Jan 14, 1:07 pm, Michael Bayer mike...@zzzcomputing.com wrote:
On Jan 14, 2011, at 12:40 PM, Randall Nortman wrote:
I'm having trouble getting mixins to play nicely with __table_args__.
Using the example I gave earlier in this thread:
On Jan 13, 1:00 pm, Michael Bayer mike...@zzzcomputing.com wrote:
On Jan 13, 2011, at 12:11 PM, Randall Nortman wrote:
[...]
namespace_id = Column(Integer, ForeignKey(Namespace.id),
nullable=False)
namespace = relationship(Namespace, backref='widgets')
name = Column(Unicode(256), nullable=False)
__table_args__ = (
UniqueConstraint(namespace_id, name),
{})
[...]
I can stick that code in a mixin class (converting everything but
'name' to be a @declared_attr), and it works just fine -- until a
class using the mixin wants to have some __table_args__ of its own.
The docs have an example of manually combining __table_args__ from
mixins in the class that uses the mixin, but I find that a little bit
ugly. In particular in this case, the mixin doesn't just use a dict
as its __table_args__, so the derived class needs to know that (or
test for that) and pull in the positional arguments and also the
dict. I don't like that, because the derived class needs to know too
much about the internals of the mixin, and I may change those
internals at some point, or else it needs some complicated code to
introspect the type of the mixin's __table_args__ (which may be a
sequence, a dict, or a @declared_attr method returning a sequence or a
dict).
Is there a nice solution that doesn't require a user of the mixin to
bend over backward to add its own __table_args__ without obliterating
the mixin's __table_args__? I would rather put complicated code in
the mixin rather than the user of the mixin, but if the user specifies
__table_args__, then the mixin's __table_args__ never seems to be
called in order to have the opportunity to do any magic.
The testing of tuple/dict as well as combining several is left up to users at
this point. I'd suggest putting that logic into a library function so that
it isn't cluttering up individual __table_args__ methods. As far as user
code knowing about the mixin, that requirement is no different from user code
that uses an __init__() being aware the requirement of __init__() for
superclasses.
It's currently an unsolved problem how conflicting tuples/dicts would be
reconciled automatically by declarative, and doing so would also make it
impossible for implementations to override its behavior, since declarative
would be forcing each individual __table_args__() to execute. So in this
area we kept the behavior the same as that which Python applies to any other
method - favoring simplicity of implementation over magic, within a process
that already is a little heavy on the magic side.
Fair enough. Here is my first cut such a library function (barely
tested):
def merge_table_args(cls, table_args):
Merges __table_args__ from one or more declarative mixins with
the given table_args argument.
positional_args = list()
dict_args = dict()
def do_merge(ta):
if hasattr(ta, 'keys'):
dict_args.update(ta)
else:
positional_args.extend(ta[:-1])
dict_args.update(ta[-1])
# Start by adding in our own table_args
do_merge(table_args)
# Now walk through the inheritance hierarchy
for base in cls.__mro__:
if issubclass(base, Base) or base is cls:
# This isn't a mixin
continue
if '__table_args__' in vars(base):
# In order to get the right class passed to a
# declared_attr on the mixin, we have to actually delete
# the existing __table_args__ and replace it with the
# mixin's.
del cls.__table_args__
cls.__table_args__ = vars(base)['__table_args__']
do_merge(cls.__table_args__)
positional_args.append(dict_args)
return tuple(positional_args)
It would be used something like this:
class Widget(Base, SomeMixin, AnotherMixin):
@declared_attr
def __table_args__(cls):
return merge_table_args(cls, (UniqueConstraint(...), {'something':
True}))
One potential semi-magical approach for doing this within declarative
might be for declarative to assume no conflict exists, and simply
combine everything in a straightforward manner (as above), and then
provide a mechanism for users to flag to declarative that a conflict
exists (or that the code otherwise wants custom behavior). In that
case, declarative would call only the most derived class's
__table_args__(), and it let implement custom logic. Perhaps
something like:
@declared_attr(auto_merge=False)
def __table_args__(cls):
...
Or:
@declared_attr
def __table_args__(cls):
...
__table_args__.auto_merge = False
--
You received this message because you are subscribed to the
Re: [sqlalchemy] dirtying attributes of a user-defined type
On Jan 13, 2011, at 3:29 PM, Michael Bayer wrote:
There's mistakes in how this is structured. UserDefinedType represents a
type object applied to a Column. The actual data handled by such a type is
not meant to be an instance of the type itself. ACLItem() here would be its
own class, and UserDefinedType would be the superclass of a class like
ACLItemType.ACLItemType() is placed on the Column, and its
bind_processor() and result_processor() deal with ACLItem objects.
Once you have the roles of type and value set up, you'd want to mixin
sqlalchemy.types.MutableType, which alerts the ORM that the value of this
type can change inline. Note that MutableType is not recommended for high
volume applications as it performs terribly, due to the need for flush() to
scan all mutables in the session for changes every time it's called - 0.7
has a new extension that allows mutable values to send change events in an
efficient manner.
Thank you for the detailed response!
I have implemented all of your advice, but have hit another roadblock-
specifically, I cannot determine how to properly cast the array type from
within the ACLItemArrayType class.
../sqlalchemy/engine/default.py, line 299, in do_execute
cursor.execute(statement, parameters)
ProgrammingError: (ProgrammingError) column aclarray is of type aclitem[] but
expression is of type text[]
LINE 1: INSERT INTO imageaclarray (aclarray) VALUES (ARRAY[E'user te...
^
HINT: You will need to rewrite or cast the expression.
'INSERT INTO imageaclarray (aclarray) VALUES (%(aclarray)s) RETURNING
imageaclarray.id' {'aclarray': ['user testrole1=rU*/testrole1', 'user
testrole2=Xc/testrole1']}
What is really need is:
insert into imageaclarray (aclarray) values (ARRAY[E'user
agentm=rU*/agentm',E'user agentm=Xc/agentm']::aclitem[]);
or
insert into imageaclarray (aclarray) values (ARRAY[E'user
agentm=rU*/agentm'::aclitem,E'user agentm=Xc/agentm'::aclitem]);
but I can't figure out how to add casting or if it is even possible... clearly
bind_processor is too late, but psycopg2 doesn't seem to offer an option to set
the oid for the container array either. Is there a method during compilation I
can use?
Cheers,
M
Code and unit test follow:
=
import sqlalchemy.types as types
import re
import sqlalchemy.exc
from sqlalchemy.dialects.postgresql import ARRAY
#include/utils/acl.h
#define ACL_ALL_RIGHTS_STR arwdDxtXUCTc
#update for SQLAlchemy 0.7:
http://www.sqlalchemy.org/docs/07/orm/extensions/mutable.html
class ACLItem(object):
def __init__(self,grantee,permissions,grantor,grant_option=False):
self.grantee = grantee
self.permissions = []
if permissions:
for p in permissions:
self.permissions.append(p)
self.grantor = grantor
self.grant_option = grant_option
def _as_pgsql_string(self):
#convert to string 'user grantee=perms/grantor'
string_perms = ''
for perm in self.permissions:
string_perms += perm
if self.grant_option:
grant_option = '*'
else:
grant_option = ''
return user %s=%s%s/%s %
(self.grantee,string_perms,grant_option,self.grantor)
@classmethod
def _from_pgsql_string(klass,aclstring):
grantee=perms*/grantor
matches = re.match('([^=]+)=([^/\*]+)(\*?)/(\w+)',aclstring)
if matches is None:
raise sqlalchemy.exc.DataError(aclstring,[],'')
grantee = matches.group(1)
permissions = matches.group(2)
grant_option = bool(len(matches.group(3)))
grantor = matches.group(4)
return ACLItem(grantee,permissions,grantor,grant_option)
def __eq__(self,other_object):
if not isinstance(other_object,self.__class__):
return False
return str(self) == str(other_object)
def has_permission(self,permission_test):
return permission_test in self.permissions
def set_permission(self,permission,on=True):
if not self.has_permission(permission) and on:
self.permissions.append(permission)
elif self.has_permission(permission) and not on:
self.permissions.remove(permission)
def clear_permissions(self):
del self.permissions[:]
def reset_with_acl(self,acl):
takes an acl and replaces its own settings with the argument settings
This is useful for cases where an acl in an array is replaced without
being creating a new aclitem so that the array order in unmodified
self.grantee = acl.grantee
self.permissions = acl.permissions
self.grantor = acl.grantor
self.grant_option = acl.grant_option
def __str__(self):
return self._as_pgsql_string()
class ACLItemArray(list):
#in an aclitem array, the
def
Re: [sqlalchemy] Re: Best way to reduce boilerplate?
On Jan 14, 2011, at 3:01 PM, Randall Nortman wrote:
On Jan 14, 1:07 pm, Michael Bayer mike...@zzzcomputing.com wrote:
On Jan 14, 2011, at 12:40 PM, Randall Nortman wrote:
The testing of tuple/dict as well as combining several is left up to users
at this point. I'd suggest putting that logic into a library function so
that it isn't cluttering up individual __table_args__ methods. As far as
user code knowing about the mixin, that requirement is no different from
user code that uses an __init__() being aware the requirement of __init__()
for superclasses.
It's currently an unsolved problem how conflicting tuples/dicts would be
reconciled automatically by declarative, and doing so would also make it
impossible for implementations to override its behavior, since declarative
would be forcing each individual __table_args__() to execute. So in this
area we kept the behavior the same as that which Python applies to any other
method - favoring simplicity of implementation over magic, within a process
that already is a little heavy on the magic side.
Fair enough. Here is my first cut such a library function (barely
tested):
def merge_table_args(cls, table_args):
Merges __table_args__ from one or more declarative mixins with
the given table_args argument.
positional_args = list()
dict_args = dict()
def do_merge(ta):
if hasattr(ta, 'keys'):
dict_args.update(ta)
else:
positional_args.extend(ta[:-1])
dict_args.update(ta[-1])
# Start by adding in our own table_args
do_merge(table_args)
# Now walk through the inheritance hierarchy
for base in cls.__mro__:
if issubclass(base, Base) or base is cls:
# This isn't a mixin
continue
if '__table_args__' in vars(base):
# In order to get the right class passed to a
# declared_attr on the mixin, we have to actually delete
# the existing __table_args__ and replace it with the
# mixin's.
del cls.__table_args__
cls.__table_args__ = vars(base)['__table_args__']
do_merge(cls.__table_args__)
positional_args.append(dict_args)
return tuple(positional_args)
It would be used something like this:
class Widget(Base, SomeMixin, AnotherMixin):
@declared_attr
def __table_args__(cls):
return merge_table_args(cls, (UniqueConstraint(...), {'something':
True}))
One potential semi-magical approach for doing this within declarative
might be for declarative to assume no conflict exists, and simply
combine everything in a straightforward manner (as above), and then
provide a mechanism for users to flag to declarative that a conflict
exists (or that the code otherwise wants custom behavior). In that
case, declarative would call only the most derived class's
__table_args__(), and it let implement custom logic. Perhaps
something like:
@declared_attr(auto_merge=False)
def __table_args__(cls):
...
Or:
@declared_attr
def __table_args__(cls):
...
__table_args__.auto_merge = False
Well I'd probably go the other way, the auto reconcile __table_args__() would
come from some option present on the base metaclass or elsewhere. that way
backwards compatibility is maintained. I'd be open to patches in this regard.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] dirtying attributes of a user-defined type
On Jan 14, 2011, at 3:26 PM, A.M. wrote:
I suspect I have to implement special converter methods with psycopg2
register_adapter/new_type. That is what I am experimenting with now.
It looks like SQLAlchemy's array type support doesn't support anything beyond
basic built-in types that accept numbers or text values. (Please do correct me
if I am wrong.) I suspect this is a limitation of the default psycopg2 array
support which does not allow passing a type along with the array during
execution. It also seems to always bind arrays inline.
I worked around this by installing type converters directly in psycopg2.
SQLAlchemy then ends up with the values as pre-cooked ACLItem and ACLItemArray
objects. From an external API standpoint, nothing changed.
One stumbling block I had was that the code sample in the docs was too slim and
there is no UserDefinedType example in the examples directory, so I spent a lot
of time in the debugger. I hope this code may help others.
Here is the code that worked:
==
import sqlalchemy.types as types
import re
import sqlalchemy.exc
from sqlalchemy.dialects.postgresql import ARRAY
from project.lib.aclitem import ACLItem,ACLItemArray
from copy import deepcopy
#include/utils/acl.h
#define ACL_ALL_RIGHTS_STR arwdDxtXUCTc
#update for SQLAlchemy 0.7:
http://www.sqlalchemy.org/docs/07/orm/extensions/mutable.html
class
ACLItemArrayType(types.MutableType,types.UserDefinedType,types.Concatenable):
def get_col_spec(self):
return 'aclitem[]'
def bind_processor(self,dialect):
return None
def result_process(self,dialect):
return None
def copy_value(self,value):
return deepcopy(value)
def compare_values(self,a,b):
return a == b
class ACLItemType(types.MutableType,types.UserDefinedType):
#class ACLItemType(types.UserDefinedType,types.MutableType): #FAIL
def get_col_spec(self):
return 'aclitem'
def bind_processor(self,dialect):
return None
def copy_value(self,value):
return deepcopy(value)
def result_processor(self,dialect,column_type):
return None
===
import re
from copy import deepcopy
class ACLItem(object):
def __init__(self,grantee,permissions,grantor,grant_option=False):
self.grantee = grantee
self.permissions = []
if permissions:
for p in permissions:
self.permissions.append(p)
self.grantor = grantor
self.grant_option = grant_option
def _as_pgsql_string(self):
#convert to string 'user grantee=perms/grantor'
string_perms = ''
for perm in self.permissions:
string_perms += perm
if self.grant_option:
grant_option = '*'
else:
grant_option = ''
return user %s=%s%s/%s %
(self.grantee,string_perms,grant_option,self.grantor)
def __deepcopy__(self,memo):
return
ACLItem(self.grantee,self.permissions,self.grantor,self.grant_option)
def __copy__(self):
return deepcopy(self)
@classmethod
def _from_pgsql_string(klass,aclstring):
grantee=perms*/grantor
matches = re.match('([^=]+)=([^/\*]+)(\*?)/(\w+)',aclstring)
if matches is None:
raise ValueError('string does not appear to represent a PostgreSQL
ACL')
grantee = matches.group(1)
permissions = matches.group(2)
grant_option = bool(len(matches.group(3)))
grantor = matches.group(4)
return ACLItem(grantee,permissions,grantor,grant_option)
def __eq__(self,other_object):
if not isinstance(other_object,self.__class__):
return False
return str(self) == str(other_object)
def has_permission(self,permission_test):
return permission_test in self.permissions
def set_permission(self,permission,on=True):
if not self.has_permission(permission) and on:
self.permissions.append(permission)
elif self.has_permission(permission) and not on:
self.permissions.remove(permission)
def clear_permissions(self):
del self.permissions[:]
def reset_with_acl(self,acl):
takes an acl and replaces its own settings with the argument settings
This is useful for cases where an acl in an array is replaced without
being creating a new aclitem so that the array order in unmodified
self.grantee = acl.grantee
self.permissions = acl.permissions
self.grantor = acl.grantor
self.grant_option = acl.grant_option
def __str__(self):
return self._as_pgsql_string()
def __repr__(self):
return ACLItem('%s',%s,'%s',%s) %
(self.grantee,self.permissions,self.grantor,self.grant_option)
class ACLItemArray(list):
#in an aclitem array, the
def aclitem_for_grantee(self,role):
for acl in self:
if role ==
[sqlalchemy] Re: SQLAlchemy 0.6.6 Released
for me its from Oracle to Netezza :) tables with 200 milion rows :) a feature im waiting for a long time is being able to drop any pyodbc connection and use it to query (orm) a db (using standard sql). Would love to see that in 0.8 :P On 14 jan., 16:19, Michael Bayer mike...@zzzcomputing.com wrote: An ETL (an acronym I keep forgetting, then I look it up, then I say, oh right I write three of those per week) is heavy on the flush side, assuming the E part is from spreadsheets or something like that. I made a few fairly vast improvements to flush() efficiency in 0.7, they're fun to watch in the logs. On Jan 14, 2011, at 4:28 AM, dusans wrote: So the C extensions would be good for someone that would use it for an ETL with a lot of data? :) On 9 jan., 17:25, Michael Bayer mike...@zzzcomputing.com wrote: I remain a little nervous about the C extensions, not as much because the current code is unreliable, but because I'd like there to be a whole lot more C code here. I've used the existing C extensions quite a bit and done tons of profiling - they really don't account for the vast majority of time spent in large ORM operations, and they don't do much for a non-ORM app that emits a lot of statements. They just cut down on an app that is non-ORM and needs to fetch very large result sets. Basically I'd need a resource that can do lots more C code for me, if not myself since it just would take a huge block of extra time for me to get into it, before I'd feel good about the C code overall and confident that I can put out releases in a timely manner which would now require 100% reliable C code. The limitations of distutils are also troubling here. In my own work app, we use pip in conjunction with a Makefile and for the SQLAlchemy install the flag is on in the Makefile. In that regard the flag being off by default doesn't feel like that big a deal to me personally. On Jan 9, 2011, at 3:33 AM, Wichert Akkerman wrote: Hi Mike, On 1/9/11 00:14 , Michael Bayer wrote: The majority of my time is now spent developing 0.7, which is nearly ready for beta releases pending a few more little features I'd like to try to get in. 0.7 is really exciting with its new event API, lots of other nice touches and of course the most radical reduction in callcounts we've had in a few years. Has been there a decision on enabling the C extensions by default in 0.7, or is that still too controversial? Regards, Wichert. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: guard all session.query with try except?
Thanks, but I don't need aysnc database access anyway. On Jan 14, 11:02 pm, Chris Withers ch...@simplistix.co.uk wrote: On 14/01/2011 06:59, can xiang wrote: Thanks for your advise. My application is a tornadoweb app. So I'm going to create Session() for each request. Tornado is an async framework, right? If so, you may be in for a bumpy ride unless you get all SQLAlchemy stuff happening in threads... cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting -http://www.simplistix.co.uk -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Best way to reduce boilerplate?
On Fri, Jan 14, 2011 at 01:07:06PM -0500, Michael Bayer wrote: It's currently an unsolved problem how conflicting tuples/dicts would be reconciled automatically by declarative, and doing so would also make it impossible for implementations to override its behavior, since declarative would be forcing each individual __table_args__() to execute. So in this area we kept the behavior the same as that which Python applies to any other method - favoring simplicity of implementation over magic, within a process that already is a little heavy on the magic side. Fair enough. One potential semi-magical approach might be for declarative to assume no conflict exists, and simply combine everything in a straightforward manner, and then provide a mechanism for users to flag to declarative that a conflict exists (or that the code otherwise wants custom behavior). In that case, declarative would call only the most derived class's __table_args__(), and it let implement custom logic. Perhaps something like: @declared_attr(auto_merge=False) def __table_args__(cls): ... Or: @declared_attr def __table_args__(cls): ... __table_args__.auto_merge = False -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
