[sqlalchemy] Re: ANNOUNCE: Spiff Guard 1.9.0
Hi! import Guard, as in the tutorial does not work. Any ideas? Easy:installing was problematic, but installing( and python setup.py install) from the download worked. I' m using windows python 2.4 Thanks! On Dec 2, 11:07 pm, Samuel [EMAIL PROTECTED] wrote: Introduction Spiff Guard is a library for implementing access lists in Python. It provides a clean and simple API and was implemented with performance and security in mind. It was originally inspired by phpGACL (http:// phpgacl.sourceforge.net/), but features an API that is significantly cleaner and easier to use. Spiff Guard is free software and distributed under the GNU GPLv2. Changes since 1.4.0: - The bad: o This release breaks API, hard. The good: o Spiff Guard is now type-aware. That means that you can create your own types and store them in the database; Spiff Guard will create an instance of the same type when you retrieve the object later. o Sections are now obsolete - instead, just use types as a section. o Spiff Guard makes now extensive use of caching. o The API is now a lot easier to use - if that is even possible. Example Code: -- class Website(ResourceGroup): pass guard = Guard(sqlalchemy_dbh) group = ResourceGroup(My Group) user= Resource(My User) website = Website(My Website) view= Action(View) write = Action(Edit) guard.grant(group, view, website) guard.grant(user, edit, website) if guard.has_permission(user, view, website): print 'Permission granted.' Dependencies - sqlalchemy (http://www.sqlalchemy.org/) Download - Tarball:http://pypi.python.org/packages/source/S/Spiff%20Guard/Spiff%20Guard-... SVN: svn checkouthttp://spiff.googlecode.com/svn/trunk/libs/Guard/ Links: --- Documentation:http://spiff.googlecode.com/svn/trunk/libs/Guard/README Example:http://spiff.googlecode.com/svn/trunk/libs/Guard/tests/DBTest.py Spiff project page:http://code.google.com/p/spiff/ Mailing list:http://groups.google.com/group/spiff-devel Bug tracker:http://code.google.com/p/spiff/issues/list Browse the source:http://spiff.googlecode.com/svn/trunk/libs/Guard/ If you have any questions, please do not hesitate to ask or file a bug. -Samuel --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: ANNOUNCE: Spiff Guard 1.9.0
On Dec 3, 3:26 pm, voltron [EMAIL PROTECTED] wrote: import Guard, as in the tutorial does not work. Any ideas? I am not familiar with Python on Windows, but does your installation directory appear in your sys.path? -Samuel --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Polymorphic from multiple columns
I'd like to make a polymorphic mapper based on two columns. Is that possible? See example code here: http://paste.pocoo.org/show/13799/ Thanks, Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Polymorphic from multiple columns
you cant do it right now. but its something we could support. its unclear to me if we should just go for composite types as the way to do it, or just use a callable. using a composite is likely cleaner and would integrate with the save process better (otherwise, the callable needs to generate the discriminator value at save time as well). On Dec 3, 2007, at 11:09 AM, Koen Bok wrote: I'd like to make a polymorphic mapper based on two columns. Is that possible? See example code here: http://paste.pocoo.org/show/13799/ Thanks, Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Polymorphic from multiple columns
is that something looking like real concrete-polymorphism? AFAIremember there was something composite there in the pattern.. the id is actualy (id,type) Michael Bayer wrote: you cant do it right now. but its something we could support. its unclear to me if we should just go for composite types as the way to do it, or just use a callable. using a composite is likely cleaner and would integrate with the save process better (otherwise, the callable needs to generate the discriminator value at save time as well). On Dec 3, 2007, at 11:09 AM, Koen Bok wrote: I'd like to make a polymorphic mapper based on two columns. Is that possible? See example code here: http://paste.pocoo.org/show/13799/ Thanks, Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Determining types of joined attributes
hi there, i'd like a way to determine if an attribute of an orm object is: a:) a sqlalchemy generated list of objects (i.e. many-to-many) b:) a single sqlalchemy joined object (i.e.one-to-many) c:) a 'scalar' loaded from the database (i.e. a string, float, integer) at present i'm copying the text generated by 'type(orm_obj.joined_list)' to determine a: and a 'type(float, int etc.)' for c:, whats left is b. this seems shakey, is there a better way? are there some 'types' defined in sqlalchemy? brendan --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Type of calculated columns
On Nov 30, 2007 9:28 PM, Michael Bayer [EMAIL PROTECTED] wrote: yes...add type_=DateTime to your coalesce() call - func.coalesce(date1, date2, type_=DateTime) This doesn't work, I'm afraid. # Table t_incident defined with sa.Column(orr_id, sa.types.Integer, primary_key=True), sa.Column(last_entry_date, sa.types.DateTime, nullable=True), sa.Column(create_date, sa.types.DateTime, nullable=False), # Standalone column definition c_activity = sa.func.coalesce( t_incident.c.last_entry_date, t_incident.c.create_date, type_=sa.types.DateTime, ).label(activity) sql = sa.select([tables.t_incident.c.orr_id, tables.c_activity, tables.t_incident.c.create_date, tables.t_incident.c.last_entry_date], limit=1) row = model.engine.execute(sql).fetchone() row (6001L, '2007-05-30 23:24:46', datetime.datetime(2005, 9, 6, 17, 14, 34), datetime.datetime(2007, 5, 30, 23, 24, 46)) The second element should be a datetime rather than a string. Using SQLAlchemy 0.4.1, Pylons dev, Python 2.5.1, Kubuntu Linux 2007.10 -- Mike Orr [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Type of calculated columns
This is on MySQL 5.0.45, BTW. -- Mike Orr [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Missing delete() attribute on orm object
hi there, i'm trying to delete an object from my database using the orm. according to 'sqlalchemy for people in a hurry docs 0.4' there is a delete() method on orm objects, so the following is possible, sample.comments[0].delete() del sample.comment[0] and this will delete the comment from the database. i don't want to enable delete-orphan as the above is just an example. i'm using sqlalchemy version 0.4.1 and python 2.4.4 regards, brendan --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Type of calculated columns
On Dec 3, 2007, at 4:05 PM, Mike Orr wrote: On Nov 30, 2007 9:28 PM, Michael Bayer [EMAIL PROTECTED] wrote: yes...add type_=DateTime to your coalesce() call - func.coalesce(date1, date2, type_=DateTime) This doesn't work, I'm afraid. oh...well actually in this case its because MSDateTime doesn't do any date/time conversion, since mysqldb returns datetime objects for us and we dont need to convert from strings...we only do it for sqlite right now. if MySQL itself is returning a datetime, then MySQLdb should as well, is it possible this is a bug on the MySQLdb side ? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Missing delete() attribute on orm object
On Dec 3, 2007, at 4:32 PM, Brendan Arnold wrote: hi there, i'm trying to delete an object from my database using the orm. according to 'sqlalchemy for people in a hurry docs 0.4' there is a delete() method on orm objects, so the following is possible, sample.comments[0].delete() del sample.comment[0] and this will delete the comment from the database. i don't want to enable delete-orphan as the above is just an example. i'm using sqlalchemy version 0.4.1 and python 2.4.4 theres no delete() on instances, even if youre using the Session.mapper feature which I dont like very much, and the pylons document should be updated if it says there is. delete() is on the Session, i.e. session.delete(someinstance). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Missing delete() attribute on orm object
thanks, i've updated the docs On Dec 3, 2007 9:35 PM, Michael Bayer [EMAIL PROTECTED] wrote: On Dec 3, 2007, at 4:32 PM, Brendan Arnold wrote: hi there, i'm trying to delete an object from my database using the orm. according to 'sqlalchemy for people in a hurry docs 0.4' there is a delete() method on orm objects, so the following is possible, sample.comments[0].delete() del sample.comment[0] and this will delete the comment from the database. i don't want to enable delete-orphan as the above is just an example. i'm using sqlalchemy version 0.4.1 and python 2.4.4 theres no delete() on instances, even if youre using the Session.mapper feature which I dont like very much, and the pylons document should be updated if it says there is. delete() is on the Session, i.e. session.delete(someinstance). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Type of calculated columns
On Dec 3, 2007 1:33 PM, Michael Bayer [EMAIL PROTECTED] wrote: On Dec 3, 2007, at 4:05 PM, Mike Orr wrote: On Nov 30, 2007 9:28 PM, Michael Bayer [EMAIL PROTECTED] wrote: yes...add type_=DateTime to your coalesce() call - func.coalesce(date1, date2, type_=DateTime) This doesn't work, I'm afraid. oh...well actually in this case its because MSDateTime doesn't do any date/time conversion, since mysqldb returns datetime objects for us and we dont need to convert from strings...we only do it for sqlite right now. if MySQL itself is returning a datetime, then MySQLdb should as well, is it possible this is a bug on the MySQLdb side ? It looks like that's the case. import MySQLdb conn = MySQLdb.connect(...) c = conn.cursor() c.execute(SELECT orr_id, COALESCE(last_entry_date, create_date) AS activity, last_entry_date, create_date FROM Incident LIMIT 1) 1L c.fetchone() (6001L, '2007-05-30 23:24:46', datetime.datetime(2007, 5, 30, 23, 24, 46), datetime.datetime(2005, 9, 6, 17, 14, 34)) I looked in my last non-SA application, and it's using the coalesce only for the select, and formatting the display date from the underlying fields. So I guess that's the best we can get unless I want to parse the string back into a date. I guess this would be a good case for an ORM property since it's read-only. -- Mike Orr [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Type of calculated columns
On Dec 3, 2007, at 4:59 PM, Mike Orr wrote: oh...well actually in this case its because MSDateTime doesn't do any date/time conversion, since mysqldb returns datetime objects for us and we dont need to convert from strings...we only do it for sqlite right now. if MySQL itself is returning a datetime, then MySQLdb should as well, is it possible this is a bug on the MySQLdb side ? It looks like that's the case. OK well, if this is just some limitation of mysql/mysqldb, i wouldnt be opposed to adding in string detection into MSDateTime. but we should check more closely with mysqldb...it seems strange that it would arbitrarily decide not to return a datetime in this case. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: concurent modification
I think the error throw of the IntegrityError is totally expensive
from both a DB perspective as well as a python perspective. if
missing data is truly so rare then it might be fine, but the
necessesity of then using SAVEPOINT seems to complicate things more
than necessary. but you'd have to benchmark different scenarios to
see which is best.
Thanks, but unfortunately, cannot get rid of tracebacks.
from threading import Thread, Lock
from sqlalchemy import create_engine, MetaData, Table, Column, types,
schema, insert, update, delete
from sqlalchemy.orm import mapper, backref, scoped_session,
sessionmaker
from sqlalchemy.exceptions import IntegrityError
metadata = MetaData()
u_table = Table('auth_user', metadata, \
Column('id', types.Integer, primary_key=True), \
Column('username', types.String(30)))
f_table = Table('fs_file', metadata, \
Column('id', types.Integer, primary_key=True), \
Column('user_id', None, schema.ForeignKey(auth_user.id)), \
Column('path', types.String, unique=True), \
Column('ls', types.PickleType, nullable=False))
class User(object):
pass
mapper(User, u_table)
mapper(Path, f_table)
session = scoped_session(sessionmaker(transactional=True,
autoflush=False))
db = create_engine(postgres://..., strategy=threadlocal)
session.configure(bind=db)
db.echo = True
class LockingManager(object):
def __init__(self):
self._lock = Lock()
if not metadata.is_bound():
metadata.bind = db
def delete(self, uid=None):
self._lock.acquire()
try:
session.execute(delete(f_table), {'user_id': uid})
session.commit() #line 88
finally:
self._lock.release()
def insert(self, uid=None, path=None, ls=None):
...
def update(self, uid=None, path=None, ls=None):
...
def select(self, username=None):
stuff = session.query(User).filter_by(username=username).first()
return stuff
def arrange(Thread):
def __init__ (self, event, manager):
Thread.__init__(self)
self.e = event
self.manager = manager
def run(self):
theone = self.manager.select(username='me')
self.manager.delete(uid=theone.id)
...
class Watch(ProcessEvent):
def __init__(self, watch_manager, lock_manager):
self._watch_manager = watch_manager
self._lock_manager = lock_manager
def process_IN_DELETE(self, event):
super(Watch, self).process_default(event)
arrange(event, self._lock_manager).start()
def main():
lm = LockingManager()
wm = WatchManager()
wm.add_watch(HOMEDIR, mask, proc_fun=Watch(wm, lm), rec=True,
auto_add=True)
...
if (__name__ == __main__):
main()
Traceback is happening on delete and always look like this:
Traceback (most recent call last):
File threading.py, line 442, in __bootstrap
self.run()
File ./camper.py, line 189, in run
walk(self.manager, theone, root)
File ./camper.py, line 101, in walk
manager.delete(uid=theone.id)
File ./camper.py, line 88, in delete
session.commit()
File /usr/lib/python2.4/site-packages/sqlalchemy/orm/scoping.py,
line 74, in do
return getattr(self.registry(), name)(*args, **kwargs)
File /usr/lib/python2.4/site-packages/sqlalchemy/orm/session.py,
line 484, in commit
self.transaction = self.transaction.commit()
File /usr/lib/python2.4/site-packages/sqlalchemy/orm/session.py,
line 211, in commit
self.session.flush()
File /usr/lib/python2.4/site-packages/sqlalchemy/orm/session.py,
line 686, in flush
self.uow.flush(self, objects)
File /usr/lib/python2.4/site-packages/sqlalchemy/orm/
unitofwork.py, line 207, in flush
flush_context.execute()
File /usr/lib/python2.4/site-packages/sqlalchemy/orm/
unitofwork.py, line 434, in execute
UOWExecutor().execute(self, head)
File /usr/lib/python2.4/site-packages/sqlalchemy/orm/
unitofwork.py, line 1053, in execute
self.execute_save_steps(trans, task)
File /usr/lib/python2.4/site-packages/sqlalchemy/orm/
unitofwork.py, line 1067, in execute_save_steps
self.save_objects(trans, task)
File /usr/lib/python2.4/site-packages/sqlalchemy/orm/
unitofwork.py, line 1058, in save_objects
task.mapper.save_obj(task.polymorphic_tosave_objects, trans)
File /usr/lib/python2.4/site-packages/sqlalchemy/orm/mapper.py,
line 1089, in save_obj
raise exceptions.ConcurrentModificationError(Updated rowcount %d
does not match number of objects updated %d % (rows, len(update)))
ConcurrentModificationError: Updated rowcount 0 does not match number
of objects updated 1
As far as I understand, one query at a time executing at a time now.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit
[sqlalchemy] Re: concurent modification
On Dec 3, 2007, at 6:33 PM, imgrey wrote:
I think the error throw of the IntegrityError is totally expensive
from both a DB perspective as well as a python perspective. if
missing data is truly so rare then it might be fine, but the
necessesity of then using SAVEPOINT seems to complicate things more
than necessary. but you'd have to benchmark different scenarios to
see which is best.
Thanks, but unfortunately, cannot get rid of tracebacks.
i can only make comments about fragments like this but i cant address
your full design issue since you havent supplied a fully working
illustration of what it is youre trying to do. the biggest issue I
can see is that the code seems to have a weak notion of explicit
transaction boundaries...its opening many sessions, one per thread
(since youre using scoped_session), and just keeping them opened, with
just one commit() when a delete is issued and thats it. all of these
sessions are going to become stale if threads continue to run...i dont
see any code here thats clearing them or closing them.If the
LockingManager lasts for a long time, it will accumulate many sessions
which will all have stale data very quickly.addtionally, youre
issuing some SQL directly to the database without notifying the
session about objects which may have been removed; thats the source of
the immediate issue youre having. even though youre executing in the
same transaction, the Session is a cache, and can't clear itself out
unless you operate on its own collection of objects.
in a nutshell, the specific error youre getting is probably because
you're essentially doing this:
sess = create_session()
myobject = sess.query(User).filter(User.id==1).one()
sess.execute(users.delete())
myobject.name = 'jack'
sess.flush() -- concurrent modification error
youre executing a SQL statement through sess.execute() but that has no
effect on the User object stored in the session..plain SQL executes
dont refresh or expire anything thats currently present in the
session. when you commit(), the underlying flush() apparently is
hitting that user, or perhaps a different one, and attempting to
update it. a concurrent modification error doesnt have anything to
do with threads or processes, only that data in your session has been
made stale by some other procedure, in this case a linear series of
steps.
so this specific issue can probably be solved like this:
def arrange(Thread):
def run(self):
theone = self.manager.select(username='me')
self.manager.delete(theone)
class LockingManager(object):
def delete(self, user):
self._lock.acquire()
try:
session.delete(user)
session.commit()
finally:
self._lock.release()
def select(self, username=None):
return session.query(User).filter_by(username=username).first()
above, the LockingManager deletes the User object, so the session is
updated with the fact that the user is now deleted. a subsequent call
to select() will return None for that user row. also note that the
locking youre doing doesnt have much effect here, without more context
it seems like its not needed and is just adding to the confusion.
another way to do this delete would be this:
session.expunge(user)
session.execute(delete(f_table), {'user_id': uid})
session.commit()
i.e., expunge the user from the session explcitly before (or even
after) issuing the SQL (but before the commit, since that will flush
the session). we do have possible plans to add mass update/delete
functionality to Session which also updates its cache automatically,
but in this case i dont see why you cant just delete() the user with
the existing ORM methods.
the main thing to be aware of here is that the Session is a cache, so
they can't be left open for a long time without issuing a clear() or
expunge() of stale data.
Its possible that you'd benefit here from using explciit transactions,
so that you dont need to be dependent on the Session in order to
commit raw SQL which youve executed. You can begin() and commit()
transactions using an Engine or a Connection...such as:
conn = db.connect()
trans = conn.begin()
session = Session(bind=conn)
# do stuff with the conn, do stuff with session
trans.commit()
Transactions should be opened and closed for individual groups of
operations. For example, if your arrange thread starts up, performs
some work, and then completes, it should be opening a transaction and/
or Session when it starts, does some work, and then closes out the
transaction/session when complete.I think the usage of thread
local convenience features here like scoped_session and
strategy=threadlocal are more than likely confusing the issue.
scoped_session is going
