Second attempt because Google Groups doesn't seems to show my question.
hi,
I'm running into an issue where unpickle a column from slqalchmey managed
database in python 2.7.5 + sqlaymchy 0.9.3 + posgresql 9.2.3
is giving different result when I use sqlalchemy to load the pickled column
or when I try to do it myself in postgresql by using a python procedure
call.
Both are using the same python enviornment.
There error is not with all rows so I assume there is some extra decoding
stuff that is going on that I'm not doing myself in postgresql procedure
call
In sqlalchemy I get this result back for some rows:
{'calculateTaxesByID': {('totalPriceEx()',): {}}, 'averageCost': {None:
Decimal('7.4838709677419355')}, 'totalPriceEx': {None: Decimal('17.50')},
'getBasePriceEx': {None: Decimal('17.50')}}
When I try to decode the same row in postgressql wiht my procedure call I
get this back
{'calculateTaxesByID': {('totalPriceEx()',): {}}}
Questions:
is SQLALchemy doing extra encoding?
is psycopg2 doing extrad encoding?
Here more details:
import pickle
class FailablePickleType(types.PickleType):
"""This syntax here is that it should replace all PickleType by this
type, that's why we sublclass PickleType and also implement
adapt.
"""
def process_bind_param(self, value, dialect):
dumps = pickle.dumps
def process(value):
try:
if value is None:
return None
return dumps(value)
except:
return None
return process
def result_processor(self, value, dialect):
loads = pickle.loads
def process(value):
try:
if value is None:
return None
return loads(value)
except (EOFError, IndexError):
return None
return None
return process
def compare_values(self, x, y):
try:
if isinstance(x, dict) and isinstance(y, dict):
return x == y
else:
return super(FailablePickleType, self).compare_values(x, y)
except:
return False
def adapt(self, impltype):
return FailablePickleType()
-----------------------------------------------
calculation_cache_transaction_table = Table(
'calculation_cache_transaction', metadata,
Column('id', Integer, primary_key=True),
Column('id_transaction', Integer, ForeignKey("transaction.id"),
index=True, nullable=False),
Column('dictionary', FailablePickleType(), nullable=True),
UniqueConstraint('id_transaction')
)
-----------------------------------------------
In my postgresql i do this:
import pickle
def co_un_pickle(value):
try:
if value is None:
return None
a = pickle.loads(value)
return a
except (EOFError, IndexError):
return None
CREATE OR REPLACE FUNCTION util.unpickle(data BYTEA) RETURNS TEXT AS
$$
from bytea import co_un_pickle
return unicode(co_un_pickle(data)).encode('raw_unicode_escape')
$$ LANGUAGE plpythonu STABLE;
SELECT util.grant_util_function('unpickle(bytea)');
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
