[sqlalchemy] Handling big Python objects
Hello list, sorry for the possibly noob question, I've googled around without much success looking for an answer. Basically, I am given a series of this huge Python class (a Simulation object), which contains an enormous amount of information - when I cPickle it (with highest protocol), it can result to files 200-250 MB in size, although rarely it can get up to 2 GB. I am looking for intelligent ways to store these objects into a database. I have to say that I don't have that much control on this Simulation class, so I can't change its internal structure - I'm just looking for a better alternative to what I am doing. So, what I am doing now is basically storing this huge object as a string. I have these two methods: import cPickle import zlib import base64 def serialize(my_simulation): my_db_object = base64.b64encode(zlib.compress(cPickle.dumps(obj, cPickle.HIGHEST_PROTOCOL))) return my_db_object def deserialize(my_db_object): my_simulation = cPickle.loads(zlib.decompress(base64.b64decode(my_db_object))) return simulation I can use them to store/retrieve this big Python classes to/from the database, but I feel it's not a particularly effective way to handle this problem. I've tried to get my head around BLOBs and LargeBinary stuff, but I'm not sure I'm on the right path here. I appreciate any suggestion on how to approach the problem, to make the storing/retrieving of these objects a bit less time/memory consuming (especially time). On a related note, I am now using MySQL as a backend - but I am open to suggestions about possible alternatives that may make this problem more tractable: I have some difficulties in installing other backends (like PostgreSQL or psycopg2), but I know I can use Access and Oracle backends. I know that Access is not particularly famous in the SQLAlchemy world. Of course, if any other backend has advantages over MySQL in dealing with this issue, I may try to convince IT to get it installed on our machines. All suggestions and comments are most welcome. Thank you in advance for your help. Andrea. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
Re: [sqlalchemy] Handling big Python objects
On Wednesday, December 3, 2014 10:42:27 PM UTC+1, Jonathan Vanasco wrote: On Wednesday, December 3, 2014 4:23:31 PM UTC-5, Ams Fwd wrote: I would recommend just storing them on disk and let the OS VMM deal with caching for speed. If you are not constrained for space I would recommend not zlib-ing it either. I'll second storing them to disk. Large object support in all the databases is a pain and not very optimal. Just pickle/unpickle a file and use the db to manage that file. Thanks to all of you who replied. A couple of issues that I'm sure I will encounter by letting the files on disk: 1. Other users can easily delete/overwrite/rename the files on disk, which is something we really, really do not want; 2. The whole point of a database was to have everything centralized in one place, not leaving the simulation files scattered around like a mess in the whole network drive; 3. As an aside, not zlib-ing the files saves about 5 seconds/simulation (over a 20 seconds save) but increases the database size by 4 times. I'll have to check if this is OK. Thank you again for your interest. Andrea. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
Re: [sqlalchemy] Handling big Python objects
Hi, On Thursday, December 4, 2014 12:02:42 AM UTC+1, Ams Fwd wrote: On 12/3/14 2:23 PM, Andrea Gavana wrote: On Wednesday, December 3, 2014 10:42:27 PM UTC+1, Jonathan Vanasco wrote: On Wednesday, December 3, 2014 4:23:31 PM UTC-5, Ams Fwd wrote: I would recommend just storing them on disk and let the OS VMM deal with caching for speed. If you are not constrained for space I would recommend not zlib-ing it either. I'll second storing them to disk. Large object support in all the databases is a pain and not very optimal. Just pickle/unpickle a file and use the db to manage that file. Thanks to all of you who replied. A couple of issues that I'm sure I will encounter by letting the files on disk: 1. Other users can easily delete/overwrite/rename the files on disk, which is something we really, really do not want; If this is windows group policies are your friends :). If this is linux, permissions with a secondary service to access the files are a decent choice. Yes, this is Windows, but no, I can't go around and tell the users that the simulation they just saved is not accessible anymore. The database is part of a much larger user interface application, where users can generate simulations and then decide whether or not they are relevant enough to be stored in the database. At a rate of 300 MB per simulation (or more), it gets quickly to the size issue. 2. The whole point of a database was to have everything centralized in one place, not leaving the simulation files scattered around like a mess in the whole network drive; The last time I did it a post processing step in my data pipeline organized the files based on a multi-level folder structure based on the first x-characters of their sha1. Again, I am dealing with non-Python people - and in general with people who are extremely good at what they do but they don't care about the overall IT architecture - as long as it works and it is recognizable from a Windows Explorer point of view. A file-based approach is unfortunately not a good option in the current setup. 3. As an aside, not zlib-ing the files saves about 5 seconds/simulation (over a 20 seconds save) but increases the database size by 4 times. I'll have to check if this is OK. To use compression or not depends on your needs. If the difference in time consumed is so stark, I would highly recommend compression. I will probably go that way, 5 seconds more or less do not make that much of a difference overall. I just wish the backends didn't complain when I pass them cPickled objects (bytes instead of strings...). Andrea. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
[sqlalchemy] Deleting an object from a database?
Hi All,
at the end, I finally cracked the problem with the tree-structured
database with SQLAlchemy. I still have however a problem.
I am using a physical database, not an in-memory one. I have
declared my Tables as follows:
trees = Table('treenodes', metadata,
Column('node_id', Integer,
Sequence('treenode_id_seq',optional=False), primary_key=True),
Column('parent_node_id', Integer,
ForeignKey('treenodes.node_id'), nullable=True),
Column('node_name', String(500), nullable=False),
Column('formattedname', String(500), nullable=False),
Column('data_ident', Integer, ForeignKey('treedata.data_id'))
)
treedata = Table(treedata, metadata,
Column('data_id', Integer, primary_key=True),
Column('value', String(100), nullable=False)
)
mapper(TreeNode, trees, properties=dict(id=trees.c.node_id,
name=trees.c.node_name,
parent_id=trees.c.parent_node_id,
children=relation(TreeNode,
cascade=all, backref=backref(parent,
remote_side=[trees.c.node_id]),
collection_class=NodeList),
data=relation(mapper(TreeData,
treedata, properties=dict(id=treedata.c.data_id)),
cascade=all,delete,delete-orphan, lazy=False))
Everything goes ok, I can save my data to a database file called
MyDatabase.db. After saving few items, the database size is about 180
Kb. So far so good. But when I delete one or more items (also if I
delete all the items) using:
self.session.delete(bad_node)
self.session.flush()
self.session.clear()
By printing the node names, I know that bad_node is no more there, but
its data seems to still live in the database. I mean, even if I delete
*all* the items, the database size is still 180 Kb, no matter how much
I flush(), clear(), close() and reopen the database.
I am sure I am missing something. Which is the usual approach to
delete *permanently* things from a database?
Thank you in advance for your suggestions.
Andrea.
Imagination Is The Only Weapon In The War Against Reality.
http://xoomer.virgilio.it/infinity77/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: Deleting an object from a database?
Hi Svilen,
first of all, thank you for your answer.
On 3/8/07, svilen wrote:
This is the db-file size, which is actual DB-implemenation detail, and
which will probably only grow up - depends on the particular db u
have. There are many strategies, like paging etc. e.g. some DBs
require a whole partition just for themselves - dont even think of
shrinking those...
See your db's doco about some way to purge, clean, vacuum the db to
get back the unused space.
Well, I am using sqlite, and at the beginning, when I create the empty
database, it has a size of 3 Kb. Then I add, let's say, 15 items with
their data and the database grows to 1 MB. Then I delete all the items
from the database, flush(), close() and whatever, and the database is
still 1 MB. I know next to nothing about SQLAlchemy and databases, and
I am sorry to ask dumb questions, but it seems to me that some space
should be cleaned...
I am surely missing something here.
Thank you for your suggestions.
Andrea.
Hi All,
at the end, I finally cracked the problem with the
tree-structured database with SQLAlchemy. I still have however a
problem.
I am using a physical database, not an in-memory one. I have
declared my Tables as follows:
trees = Table('treenodes', metadata,
Column('node_id', Integer,
Sequence('treenode_id_seq',optional=False), primary_key=True),
Column('parent_node_id', Integer,
ForeignKey('treenodes.node_id'), nullable=True),
Column('node_name', String(500), nullable=False),
Column('formattedname', String(500), nullable=False),
Column('data_ident', Integer,
ForeignKey('treedata.data_id')) )
treedata = Table(treedata, metadata,
Column('data_id', Integer, primary_key=True),
Column('value', String(100), nullable=False)
)
mapper(TreeNode, trees, properties=dict(id=trees.c.node_id,
name=trees.c.node_name,
parent_id=trees.c.parent_node_id, children=relation(TreeNode,
cascade=all, backref=backref(parent,
remote_side=[trees.c.node_id]),
collection_class=NodeList),
data=relation(mapper(TreeData, treedata,
properties=dict(id=treedata.c.data_id)),
cascade=all,delete,delete-orphan, lazy=False))
Everything goes ok, I can save my data to a database file called
MyDatabase.db. After saving few items, the database size is about
180 Kb. So far so good. But when I delete one or more items (also
if I delete all the items) using:
self.session.delete(bad_node)
self.session.flush()
self.session.clear()
By printing the node names, I know that bad_node is no more there,
but its data seems to still live in the database. I mean, even if I
delete *all* the items, the database size is still 180 Kb, no
matter how much I flush(), clear(), close() and reopen the
database.
I am sure I am missing something. Which is the usual approach to
delete *permanently* things from a database?
Thank you in advance for your suggestions.
Andrea.
Imagination Is The Only Weapon In The War Against Reality.
http://xoomer.virgilio.it/infinity77/
--
Andrea.
Imagination Is The Only Weapon In The War Against Reality.
http://xoomer.virgilio.it/infinity77/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: Deleting an object from a database?
Hi Michael,
On 3/8/07, Michael Bayer wrote:
have you verified that the tables have been deleted from ? if the
filesize doesnt shrink, that may be an artifact of sqlite's
implementation.
Thank you for your answer. Well, I can reproduce it in a small Python
script that I attach to my email. At the end of the script there is
this declaration:
# AG: Just modify this variable, the first time to create a new
# database use True, then use False to see what happens by
# loading the existing database
newDataBase = True
If you run the sample with newDataBase=True, a new database called
tutorial_modified.db is created, and it is around 120 Kb in size.
Changing newDataBase to False, will cause the script to load the
existing database.
At this point, in the script I do this:
self.session.clear()
# This selects the root node:
t = self.session.query(TreeNode).select(TreeNode.c.name==rootnode)[0]
# This should delete everything from the session/db, shouldn't it?
self.session.delete(t)
del t
self.session.flush()
But the database is still 120 Kb in size, no shrinkage of 1 byte...
The script itself will print out the database size using os.stat.
I am really lost... does anyone know what I am doing wrong?
Thank you for your suggestions.
Andrea.
Imagination Is The Only Weapon In The War Against Reality.
http://xoomer.virgilio.it/infinity77/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
a basic Adjacency List model tree.
import os
from sqlalchemy import *
from sqlalchemy.util import OrderedDict
class TreeData(object):
def __init__(self, value=None):
self.id = None
self.value = value
def __repr__(self):
return TreeData(%s, %s) % (repr(self.id), repr(self.value))
class NodeList(OrderedDict):
subclasses OrderedDict to allow usage as a list-based property.
def append(self, node):
self[node.name] = node
def __iter__(self):
return iter(self.values())
class TreeNode(object):
a rich Tree class which includes path-based operations
def __init__(self, name):
self.children = NodeList()
self.name = name
self.parent = None
self.id = None
self.parent_id = None
self.value = None
def setdata(self, data):
self.data = data
def getdata(self):
return self.data
def append(self, node):
if isinstance(node, str):
node = TreeNode(node)
node.parent = self
self.children.append(node)
def __repr__(self):
return self._getstring(0, False)
def __str__(self):
return self._getstring(0, False)
def _getstring(self, level, expand = False):
s = (' ' * level) + %s (%s,%s, %d) % (self.name,
self.id,self.parent_id,id(self)) + '\n'
if expand:
s += ''.join([n._getstring(level+1, True) for n in
self.children.values()])
return s
def print_nodes(self):
return self._getstring(0, True)
class TheEngine(object):
def __init__(self, newDataBase=True):
if newDataBase and os.path.isfile(tutorial_modified.db):
os.remove(tutorial_modified.db)
self.engine = create_engine('sqlite:///tutorial_modified.db',
echo=False)
metadata = BoundMetaData(self.engine)
trees = Table('treenodes', metadata,
Column('node_id', Integer,
Sequence('treenode_id_seq',optional=False), primary_key=True),
Column('parent_node_id', Integer, ForeignKey('treenodes.node_id'),
nullable=True),
Column('node_name', String(50), nullable=False),
Column('data_ident', Integer, ForeignKey('treedata.data_id'))
)
treedata = Table(treedata, metadata,
Column('data_id', Integer, primary_key=True),
Column('value', String(100), nullable=False)
)
mapper(TreeNode, trees, properties=dict(id=trees.c.node_id,
name=trees.c.node_name,
parent_id=trees.c.parent_node_id,
children=relation(TreeNode,
cascade=all, backref=backref(parent, remote_side=[trees.c.node_id]),
collection_class=NodeList),
data=relation(mapper(TreeData,
treedata, properties=dict(id=treedata.c.data_id)),
cascade=delete,delete-orphan,save-update, lazy=False))
)
metadata.create_all()
self.session = create_session()
if newDataBase:
[sqlalchemy] Re: Deleting an object from a database?
Hi Michael and All, thank you for your detailed answer, and to have tried the demo I have attached. I understand what you meant and you are perfectly right, but if I can bother you with another small question: On 3/8/07, Michael Bayer wrote: z-eeks-Computer:~/dev/sqlalchemy classic$ sqlite3 tutorial_modified.db SQLite version 3.3.13 Enter .help for instructions sqlite select * from treenodes; sqlite select * from treedata; sqlite so, its all deleted. filesize is irrelevant. Uhm, really? What happens if Windows is saying that the filesize is 4 GB of empty things? I told you I know next to nothing about database, but it seems a bit odd to me that the file size constantly increases even if I delete things from the database... Please forgive my newbieness and my way to write, I don't want to sound disrespectful, I am just exposing my doubts and my questions. I believe SQLAlchemy is the easiest Python-database-library to use, and for this reason I would like to avoid making stupid mistakes which would be much more complicated to fix later. Thank you for your answer. Andrea. Imagination Is The Only Weapon In The War Against Reality. http://xoomer.virgilio.it/infinity77/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Deleting an object from a database?
Hi Michael, On 3/8/07, Michael Bayer wrote: but the point is, and why its irrelvant to me as the maintainer of SQLAlchemy, is that this is totally an issue with sqlite, and has nothing to do with SQLAlchemy. you should ask on their mailing list about this particular behavior. Sorry, I didn't understand it was directly related to sqlite, I thought it was a general behavior of all databases, but obviously I was wrong. I have chose sqlite because it's already there in Python 2.5, but I'll try with other databases to see what happens. Thank you for your answer. Andrea. Imagination Is The Only Weapon In The War Against Reality. http://xoomer.virgilio.it/infinity77/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Deleting an object from a database?
Hi Michael and All, On 3/8/07, Michael Bayer wrote: but the point is, and why its irrelvant to me as the maintainer of SQLAlchemy, is that this is totally an issue with sqlite, and has nothing to do with SQLAlchemy. you should ask on their mailing list about this particular behavior. I was able to get a very useful answer from the pysqlite mailing list. Just for the records, if someone else will have the same problem in the future (very unlikely, but one never knows...), it was enough to call the VACUUM command for sqlite using an SQLAlchemy statement: self.engine.execute(VACUUM) After having deleted the nodes. I don't know if this is the most correct way to proceed, but the command has shrunk back the database file size to 3 Kb, which was the original file size of the empty database. Thank you for your help. Andrea. Imagination Is The Only Weapon In The War Against Reality. http://xoomer.virgilio.it/infinity77/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Compressing cPickled objects == store in DB
Hi All,
I was wondering if it is possible to use zlib.compress() to
compress a cPickled object and store it into a database, and then
retrieve it using zlib.decompress(). I have tried for a while, but I
keep getting errors like:
snip
File build\bdist.win32\egg\sqlalchemy\engine\base.py, line 369, in _execute
sqlalchemy.exceptions.SQLError: (OperationalError) Could not decode to UTF-8 col
' 'SELECT treenodes.node_name AS treenodes_node_name, treenodes.parent_node_id A
S treenodes_parent_node_id, treenodes.node_id AS treenodes_node_id, treedata_f69
f.data_id AS treedata_f69f_data_id, treedata_f69f.value AS treedata_f69f_value,
treenodes.data_ident AS treenodes_data_ident \nFROM treenodes LEFT OUTER JOIN tr
eedata AS treedata_f69f ON treedata_f69f.data_id = treenodes.data_ident \nWHERE
treenodes.node_name = ? ORDER BY treenodes.oid, treedata_f69f.oid' ['rootnode']
I assume something like that is not possible, although it would be
handy to reduce the size of stored objects in a database... I attach a
small demo if someone is willing to try it. As usual, just change the
variable newDataBase=True for the first run (no errors, but database
looks empty, the size is only 3 Kb), and then change to
newDataBase=False.
Any thought?
Andrea.
Imagination Is The Only Weapon In The War Against Reality.
http://xoomer.virgilio.it/infinity77/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
a basic Adjacency List model tree.
import os
import cPickle
import zlib
from sqlalchemy import *
from sqlalchemy.util import OrderedDict
class TreeData(object):
def __init__(self, value=None):
self.id = None
self.value = value
def __repr__(self):
return TreeData(%s, %s) % (repr(self.id), repr(self.value))
class NodeList(OrderedDict):
subclasses OrderedDict to allow usage as a list-based property.
def append(self, node):
self[node.name] = node
def __iter__(self):
return iter(self.values())
class TreeNode(object):
a rich Tree class which includes path-based operations
def __init__(self, name):
self.children = NodeList()
self.name = name
self.parent = None
self.id = None
self.parent_id = None
self.value = None
def setdata(self, data):
self.data = data
def getdata(self):
return self.data
def append(self, node):
if isinstance(node, str):
node = TreeNode(node)
node.parent = self
self.children.append(node)
def __repr__(self):
return self._getstring(0, False)
def __str__(self):
return self._getstring(0, False)
def _getstring(self, level, expand = False):
s = (' ' * level) + %s (%s,%s, %d) % (self.name,
self.id,self.parent_id,id(self)) + '\n'
if expand:
s += ''.join([n._getstring(level+1, True) for n in
self.children.values()])
return s
def print_nodes(self):
return self._getstring(0, True)
class TheEngine(object):
def __init__(self, newDataBase=True):
if newDataBase and os.path.isfile(tutorial_modified.db):
os.remove(tutorial_modified.db)
self.engine = create_engine('sqlite:///tutorial_modified.db',
echo=False)
metadata = BoundMetaData(self.engine)
trees = Table('treenodes', metadata,
Column('node_id', Integer,
Sequence('treenode_id_seq',optional=False), primary_key=True),
Column('parent_node_id', Integer, ForeignKey('treenodes.node_id'),
nullable=True),
Column('node_name', String(50), nullable=False),
Column('data_ident', Integer, ForeignKey('treedata.data_id'))
)
treedata = Table(treedata, metadata,
Column('data_id', Integer, primary_key=True),
Column('value', String(100), nullable=False)
)
mapper(TreeNode, trees, properties=dict(id=trees.c.node_id,
name=trees.c.node_name,
parent_id=trees.c.parent_node_id,
children=relation(TreeNode,
cascade=all, backref=backref(parent, remote_side=[trees.c.node_id]),
collection_class=NodeList),
data=relation(mapper(TreeData,
treedata, properties=dict(id=treedata.c.data_id)),
cascade=delete,delete-orphan,save-update, lazy=False))
)
metadata.create_all()
self.session = create_session()
if
[sqlalchemy] Re: Problems with select() :-(
Hi Jonathan and All, first of all, thank you for your answer. On 3/6/07, Jonathan Ellis wrote: it sounds like you inserted a bunch of new objects, but didn't flush, so select() doesn't see them. get() does see it because it checks the identity map before querying the db. Well, at the beginning this is what I thought, but I put flush() everywhere and I still get the same result. So, I thought to modify the byroot_tree.py demo in the examples/adjacencytree directory to show a couple of issues I have been fighting for 2 days. The problems you will see are: 1) No matter how/when/where I flush(), select returns always an empty list (see the attached file when you run the simple demo); 2) Using a physical database (not using an in-memory database), baffles me: in the attached file, at the bottom, just set the variable: newDataBase = True The first time to create the database: this just creates few random nodes and saves them into the database. Then, set: newDataBase = False To try to load the data from the database. I get an impossible error from the TreeLoader class: Traceback (most recent call last): File C:\Documents and Settings\gavana\Desktop\SQLAlchemy-0.3.5\SQLAlchemy-0.3 .5\examples\adjacencytree\byroot_tree_1.py, line 206, in module main() File C:\Documents and Settings\gavana\Desktop\SQLAlchemy-0.3.5\SQLAlchemy-0.3 .5\examples\adjacencytree\byroot_tree_1.py, line 203, in main engineclass = TheEngine(newDataBase) File C:\Documents and Settings\gavana\Desktop\SQLAlchemy-0.3.5\SQLAlchemy-0.3 .5\examples\adjacencytree\byroot_tree_1.py, line 157, in __init__ self.LoadNodes() File C:\Documents and Settings\gavana\Desktop\SQLAlchemy-0.3.5\SQLAlchemy-0.3 .5\examples\adjacencytree\byroot_tree_1.py, line 193, in LoadNodes parentnode = query.get(ii) File build\bdist.win32\egg\sqlalchemy\orm\query.py, line 61, in get File build\bdist.win32\egg\sqlalchemy\orm\query.py, line 376, in _get File build\bdist.win32\egg\sqlalchemy\orm\query.py, line 384, in _select_sta tement File build\bdist.win32\egg\sqlalchemy\orm\query.py, line 316, in execute File build\bdist.win32\egg\sqlalchemy\orm\query.py, line 335, in instances File build\bdist.win32\egg\sqlalchemy\orm\mapper.py, line 1258, in _instance File build\bdist.win32\egg\sqlalchemy\orm\mapper.py, line 1439, in append_re sult File build\bdist.win32\egg\sqlalchemy\orm\mapper.py, line 1456, in _do File build\bdist.win32\egg\sqlalchemy\orm\mapper.py, line 1439, in append_re sult File build\bdist.win32\egg\sqlalchemy\orm\mapper.py, line 1456, in _do File C:\Documents and Settings\gavana\Desktop\SQLAlchemy-0.3.5\SQLAlchemy-0.3 .5\examples\adjacencytree\byroot_tree_1.py, line 91, in append_result parentnode = selectcontext.identity_map[mapper.identity_key(instance.parent_ id)] KeyError: (class '__main__.TreeNode', (1,), None) I know I am doing something really stupid, but I don't know how to fix it. I obviously need a physical database, not an in-memory, and I would like to be able to load the data after I saved them. I am sorry for my poor knowledge of SQLAlchemy, I just started. Thank you for your suggestions. Andrea. Imagination Is The Only Weapon In The War Against Reality. http://xoomer.virgilio.it/infinity77/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~--- a more advanced example of basic_tree.py. treenodes can now reference their root node, and introduces a new selection method which selects an entire tree of nodes at once, taking advantage of a custom MapperExtension to assemble incoming nodes into their correct structure. import os from sqlalchemy import * from sqlalchemy.util import OrderedDict class TreeData(object): def __init__(self, value=None): self.id = None self.value = value def __repr__(self): return TreeData(%s, %s) % (repr(self.id), repr(self.value)) class NodeList(OrderedDict): subclasses OrderedDict to allow usage as a list-based property. def append(self, node): self[node.name] = node def __iter__(self): return iter(self.values()) class TreeNode(object): a hierarchical Tree class, which adds the concept of a root node. The root is the topmost node in a tree, or in other words a node whose parent ID is NULL. All child nodes that are decendents of a particular root, as well as a root node itself, reference this root node. this is useful as a way to identify all nodes in a tree as belonging to a single identifiable root. Any node can return its root node and therefore the tree that it belongs to, and entire
