[sqlalchemy] Creating a custom type
I'm trying to build a custom type [1] to manage the bcrypt hashes [2]. --- from bcrypt_wrap import password from sqlalchemy import types class Bcrypt(types.TypeDecorator): Stores a bcrypt hash of a password. impl = types.String #(60) hasher = password.Password() def process_bind_param(self, value, dialect): return hasher.create(value) --- And I've any doubts: 1) Since that the hash length is always 60, is there that use the next? impl = types.String(60) 2) The bcryptWrap API [3] (line 53) lets call to 'create' with arguments 'cost' and 'admin' (boolean). Then, a) I would that the cost could be passed from the column definition, i.e. password = sqlalchemy.Column(types.Bcrypt, cost=12) or password = sqlalchemy.Column(types.Bcrypt(cost=12)) b) This would be more hard, but I would to pass the 'admin' argument when an object is created u = model.User() u.login = u'foo' u.password = u'bar' u.admin = True And sou could be call: return hasher.create(value, admin=True) [1] http://www.sqlalchemy.org/docs/05/reference/sqlalchemy/types.html#custom-types [2] http://pypi.python.org/pypi/bcryptWrap/ [3] http://www.bitbucket.org/ares/bcryptwrap/src/tip/lib/bcrypt_wrap/password.py --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Creating a custom type
On Dec 21, 2008, at 6:53 AM, Kless wrote:
I'm trying to build a custom type [1] to manage the bcrypt hashes [2].
---
from bcrypt_wrap import password
from sqlalchemy import types
class Bcrypt(types.TypeDecorator):
Stores a bcrypt hash of a password.
impl = types.String #(60)
hasher = password.Password()
def process_bind_param(self, value, dialect):
return hasher.create(value)
---
And I've any doubts:
1) Since that the hash length is always 60, is there that use the
next?
impl = types.String(60)
2) The bcryptWrap API [3] (line 53) lets call to 'create' with
arguments 'cost' and 'admin' (boolean). Then,
a) I would that the cost could be passed from the column definition,
i.e.
password = sqlalchemy.Column(types.Bcrypt, cost=12)
or
password = sqlalchemy.Column(types.Bcrypt(cost=12))
b) This would be more hard, but I would to pass the 'admin' argument
when an object is created
u = model.User()
u.login = u'foo'
u.password = u'bar'
u.admin = True
And sou could be call:
return hasher.create(value, admin=True)
the TypeDecorator's __init__ method just calls the impl class
immediately, so the best approach is like:
class Bcrypt(types.TypeDecorator):
Stores a bcrypt hash of a password.
impl = types.String
hasher = password.Password()
def __init__(self, cost):
self.cost = cost
types.TypeDecorator.__init__(self, 60)
def process_bind_param(self, value, dialect):
return hasher.create(value)
then you can instantate like:
Column('foo', Bcrypt(cost=12))
For b, the type object and the ExecutionContext which ultimately calls
its process_bind_param method are unaware of the ORM or the ORM-
specific context in which its called during INSERT/UPDATE. If you
wanted to keep the logic within your type like that you'd have to
integrate to a thread-local variable that is configured within an ORM
plugin, something like:
hasher_status = threading.local()
class MyMapperExt(MapperExtension):
def before_insert(self, ...):
hasher_status.admin = instance.admin
def after_insert(self, ...)
del hasher_status.admin
mapper(MyClass, table, ext=MyMapperExt)
class Bcrypt(types.TypeDecorator):
...
def process_bind_param(self, value, dialect):
return hasher.create(value, getattr(hasher_status, 'admin',
False))
Alternatively, you could keep the logic within the ORM using either a
valdiator or a descriptor, as described at
http://www.sqlalchemy.org/docs/05/mappers.html#changing-attribute-behavior
. It depends on if you'd like your Bcrypt type to work with direct
SQL expression language use or not.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: Creating a custom type
Thank you for detailed answer.
Here are any thoughts:
1) The before_insert was being omitted while I was inserting records
using an insert statement --that's logical because it was bypassing
the ORM--.
2) It's necessary to use *session.commit()* after of each record
because else it will have the value of 'instance.admin' of the last
record which will be used for all records to commit.
3) I believe that it isn't necessary use *after_insert*.
On 21 dic, 17:00, Michael Bayer mike...@zzzcomputing.com wrote:
On Dec 21, 2008, at 6:53 AM, Kless wrote:
I'm trying to build a custom type [1] to manage the bcrypt hashes [2].
---
from bcrypt_wrap import password
from sqlalchemy import types
class Bcrypt(types.TypeDecorator):
Stores a bcrypt hash of a password.
impl = types.String #(60)
hasher = password.Password()
def process_bind_param(self, value, dialect):
return hasher.create(value)
---
And I've any doubts:
1) Since that the hash length is always 60, is there that use the
next?
impl = types.String(60)
2) The bcryptWrap API [3] (line 53) lets call to 'create' with
arguments 'cost' and 'admin' (boolean). Then,
a) I would that the cost could be passed from the column definition,
i.e.
password = sqlalchemy.Column(types.Bcrypt, cost=12)
or
password = sqlalchemy.Column(types.Bcrypt(cost=12))
b) This would be more hard, but I would to pass the 'admin' argument
when an object is created
u = model.User()
u.login = u'foo'
u.password = u'bar'
u.admin = True
And sou could be call:
return hasher.create(value, admin=True)
the TypeDecorator's __init__ method just calls the impl class
immediately, so the best approach is like:
class Bcrypt(types.TypeDecorator):
Stores a bcrypt hash of a password.
impl = types.String
hasher = password.Password()
def __init__(self, cost):
self.cost = cost
types.TypeDecorator.__init__(self, 60)
def process_bind_param(self, value, dialect):
return hasher.create(value)
then you can instantate like:
Column('foo', Bcrypt(cost=12))
For b, the type object and the ExecutionContext which ultimately calls
its process_bind_param method are unaware of the ORM or the ORM-
specific context in which its called during INSERT/UPDATE. If you
wanted to keep the logic within your type like that you'd have to
integrate to a thread-local variable that is configured within an ORM
plugin, something like:
hasher_status = threading.local()
class MyMapperExt(MapperExtension):
def before_insert(self, ...):
hasher_status.admin = instance.admin
def after_insert(self, ...)
del hasher_status.admin
mapper(MyClass, table, ext=MyMapperExt)
class Bcrypt(types.TypeDecorator):
...
def process_bind_param(self, value, dialect):
return hasher.create(value, getattr(hasher_status, 'admin',
False))
Alternatively, you could keep the logic within the ORM using either a
valdiator or a descriptor, as described
athttp://www.sqlalchemy.org/docs/05/mappers.html#changing-attribute-beh...
. It depends on if you'd like your Bcrypt type to work with direct
SQL expression language use or not.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: Creating a custom type
oh, yeah for that recipe you'd have to use a flag on the mapper()
called batch=False.But don't use that, its inefficient.
So you can also create your type to detect a special value from the
mapper extension:
def before_insert(self, ):
instance.password = (instance.password, True)
class MyType(...):
...
def process_bind_param(self, value, dialect):
if isinstance(value, tuple):
return hasher.create(value[0], value[1])
But since this is really an instance-level business rule, a straight
descriptor and no custom type is definitely how I'd go on this one.
On Dec 21, 2008, at 6:47 PM, Kless wrote:
Thank you for detailed answer.
Here are any thoughts:
1) The before_insert was being omitted while I was inserting records
using an insert statement --that's logical because it was bypassing
the ORM--.
2) It's necessary to use *session.commit()* after of each record
because else it will have the value of 'instance.admin' of the last
record which will be used for all records to commit.
3) I believe that it isn't necessary use *after_insert*.
On 21 dic, 17:00, Michael Bayer mike...@zzzcomputing.com wrote:
On Dec 21, 2008, at 6:53 AM, Kless wrote:
I'm trying to build a custom type [1] to manage the bcrypt hashes
[2].
---
from bcrypt_wrap import password
from sqlalchemy import types
class Bcrypt(types.TypeDecorator):
Stores a bcrypt hash of a password.
impl = types.String #(60)
hasher = password.Password()
def process_bind_param(self, value, dialect):
return hasher.create(value)
---
And I've any doubts:
1) Since that the hash length is always 60, is there that use the
next?
impl = types.String(60)
2) The bcryptWrap API [3] (line 53) lets call to 'create' with
arguments 'cost' and 'admin' (boolean). Then,
a) I would that the cost could be passed from the column
definition,
i.e.
password = sqlalchemy.Column(types.Bcrypt, cost=12)
or
password = sqlalchemy.Column(types.Bcrypt(cost=12))
b) This would be more hard, but I would to pass the 'admin'
argument
when an object is created
u = model.User()
u.login = u'foo'
u.password = u'bar'
u.admin = True
And sou could be call:
return hasher.create(value, admin=True)
the TypeDecorator's __init__ method just calls the impl class
immediately, so the best approach is like:
class Bcrypt(types.TypeDecorator):
Stores a bcrypt hash of a password.
impl = types.String
hasher = password.Password()
def __init__(self, cost):
self.cost = cost
types.TypeDecorator.__init__(self, 60)
def process_bind_param(self, value, dialect):
return hasher.create(value)
then you can instantate like:
Column('foo', Bcrypt(cost=12))
For b, the type object and the ExecutionContext which ultimately
calls
its process_bind_param method are unaware of the ORM or the ORM-
specific context in which its called during INSERT/UPDATE. If you
wanted to keep the logic within your type like that you'd have to
integrate to a thread-local variable that is configured within an ORM
plugin, something like:
hasher_status = threading.local()
class MyMapperExt(MapperExtension):
def before_insert(self, ...):
hasher_status.admin = instance.admin
def after_insert(self, ...)
del hasher_status.admin
mapper(MyClass, table, ext=MyMapperExt)
class Bcrypt(types.TypeDecorator):
...
def process_bind_param(self, value, dialect):
return hasher.create(value, getattr(hasher_status, 'admin',
False))
Alternatively, you could keep the logic within the ORM using either a
valdiator or a descriptor, as described
athttp://www.sqlalchemy.org/docs/05/mappers.html#changing-attribute-beh
...
. It depends on if you'd like your Bcrypt type to work with direct
SQL expression language use or not.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] retrying queries and 'Lost connection to MySQL server'
I occasionally have a query fail with 'Lost connection to MySQL server during query' which gets converted into a sqlalchemy.exceptions.OperationalError. I have not been able to figure out why it happens, but the server should always be available. I would like to tell sqlalchemy that if a query fails with this error, it should wait a few seconds and then retry the query (and probably give up if it fails again). Does SQLA provide some sort of hooks that would allow me to do this without gnarly monkey patching? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: retrying queries and 'Lost connection to MySQL server'
On Dec 21, 2008, at 11:24 PM, Bobby Impollonia wrote: I occasionally have a query fail with 'Lost connection to MySQL server during query' which gets converted into a sqlalchemy.exceptions.OperationalError. I have not been able to figure out why it happens, but the server should always be available. I would like to tell sqlalchemy that if a query fails with this error, it should wait a few seconds and then retry the query (and probably give up if it fails again). Does SQLA provide some sort of hooks that would allow me to do this without gnarly monkey patching? you'd have to organize your code such that the desired operation can be attempted again when this exception is raised. This is a pretty tough road to travel, though, since if the connection is lost, so is your entire transaction and everything you've loaded/ persisted within it. A better approach would be to isolate the cause of the error.This error is commonly caused by a MySQL client timeout (usually on a connection that's been idle for 8 hours) and is allevated using the pool_recycle=some number of seconds option. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: retrying queries and 'Lost connection to MySQL server'
This code isn't using transactions so retrying a failed query should be as simple as creating a new connection to replace the failed one and executing the query again. Still, I would much prefer to figure out the real cause, as you say. I had sort of given up on that because after a little while researching this error, I couldn't find much helpful info. It's hard to debug because the issue happens in a daily cron job, but it happens less than once a month and the rest of the time everything works fine. I have no way of consistently reproducing the problem or knowing if I've fixed it. I'm pretty sure there is no way that 8 hours could have gone by between the last query and the one that blew up. The basic structure of the cron job is: 1) It start up, does some sql stuff. 2) It forks a worker process using the python processing module. 3a) The worker calls metadata.bind.dispose() so that it won't try to reuse the connection it inherited from the parent. Worker then does some sql stuff. Worker always finishes successfully. 3b) Parent process goes into a loop doing sql stuff. Parent usually finishes successfully, but occasionally dies with the aforementioned MySQL error. I can't tell from the traceback whether it happens during the first iteration of the loop immediately after spawning the child or if it happens later. In principle, this structure is safe, right? 3a and 3b are happening in parallel, so it is indeterminate whether the worker calls dispose() before or during the sql stuff going on in the parent, but that shouldn't mater, right? Is it possible that the call to dispose() is somehow closing the connection in a way that sabotages the parent? On Sun, Dec 21, 2008 at 11:32 PM, Michael Bayer mike...@zzzcomputing.com wrote: On Dec 21, 2008, at 11:24 PM, Bobby Impollonia wrote: I occasionally have a query fail with 'Lost connection to MySQL server during query' which gets converted into a sqlalchemy.exceptions.OperationalError. I have not been able to figure out why it happens, but the server should always be available. I would like to tell sqlalchemy that if a query fails with this error, it should wait a few seconds and then retry the query (and probably give up if it fails again). Does SQLA provide some sort of hooks that would allow me to do this without gnarly monkey patching? you'd have to organize your code such that the desired operation can be attempted again when this exception is raised. This is a pretty tough road to travel, though, since if the connection is lost, so is your entire transaction and everything you've loaded/ persisted within it. A better approach would be to isolate the cause of the error.This error is commonly caused by a MySQL client timeout (usually on a connection that's been idle for 8 hours) and is allevated using the pool_recycle=some number of seconds option. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
