On 09/23/2016 10:18 AM, HP3 wrote:
Hello all,
Couple of weeks back (see [*]), while discussing that history_meta.py
performs an update and an insert for each session.dirty object whenever
session.flush() happens, Simon suggested the following solution to
coalesce all changes within the same transaction into a single insert
and hence a single version increase:
1. In a before_flush hook, check to see if the object has really
changed (using the code from create_version). If it has, save it away in
a set somewhere.
2. In a before_commit hook, iterate over that set and create the
actual history entries.
I went ahead and produced a proof of concept of what he suggested.
Although I have not fully tested it, it seems to work well.
Nonetheless, I wanted to 2x check with you couple of points:
a) For step 1, I used `session.info` to stash `history_objects` - a
dictionary containing the newly created XYZHistory instances keyed by
their "parent" XYZ instance's id and version. Is that a good place to
store them? I guess I would need to "clear" it up on a
`session.rollback` and `session.commit` ?
session.info is great for that. If you want to clear it out when
transactions end, there's a few events you can use for that and
after_transaction_end might be a good one.
http://docs.sqlalchemy.org/en/latest/orm/events.html?highlight=after_transaction_end#sqlalchemy.orm.events.SessionEvents.after_transaction_end
b) For step 2, increase each XYZ instance's version `before_commit`
right when any newly added XYZHistory instance is being added to the
session.
before_commit() is invoked before the final flush() so this is a great
place to do that.
Any thoughts? suggestions? comments?
Thanks in advance
Here is a sneak peak of the changes:
```python
# history_meta.py
@event.listens_for(session, 'before_flush')
def versioned_session_before_flush_handler(session):
if "history-objects" not in session.info:
history_objects = dict()
session.info["history-objects"] = history_objects
...
@event.listens_for(session, 'before_commit')
def versioned_session_before_commit_handler(session):
for hist in history_objects.values():
session.add(hist)
vobj =
session.query(inspect(hist).class_.__versioning_cls__).get(hist.id) #
__versioning_cls__ is the class with the Versioned mixin
vobj.version = hist.version + 1
# near the bottom of `create_version`
...
hist_pk = (inspect(obj).identity, obj.version)
hist = history_objects.get(hist_pk, None)
if hist is None:
hist = hist_cls()
...
```
[*]
https://groups.google.com/forum/#!topic/sqlalchemy/jlEqIidnW_s
https://groups.google.com/forum/#!topic/sqlalchemy/b0tEjftCWkM
Here is the diff (against rel_0_9_8)
$ git diff rel_0_9_8..test_version_relationship_0_9_8 --
examples/versioned_history/history_meta.py
diff --git a/examples/versioned_history/history_meta.py
b/examples/versioned_history/history_meta.py
index f9e979a..84ba87b 100644
--- a/examples/versioned_history/history_meta.py
+++ b/examples/versioned_history/history_meta.py
@@ -7,6 +7,10 @@from sqlalchemy import Table, Column,
ForeignKeyConstraint, Integer, DateTime
from sqlalchemy import event
import datetime
from sqlalchemy.orm.properties import RelationshipProperty
+from sqlalchemy import inspect
+
+import logging
+log = logging.getLogger("History-Meta")
def col_references_table(col, table):
for fk in col.foreign_keys:
@@ -68,9 +72,11 @@def _history_mapper(local_mapper):
# "changed" column stores the UTC timestamp of when the
# history row was created.
# This column is optional and can be omitted.
+ """
cols.append(Column('changed', DateTime,
default=datetime.datetime.utcnow,
info=version_meta))
+ """
if super_fks:
cols.append(ForeignKeyConstraint(*zip(*super_fks)))
@@ -95,6 +101,8 @@def _history_mapper(local_mapper):
bases = local_mapper.base_mapper.class_.__bases__
versioned_cls = type.__new__(type, "%sHistory" % cls.__name__,
bases, {})
+ versioned_cls.__versioning_cls__ = cls
+
m = mapper(
versioned_cls,
table,
@@ -191,20 +210,46 @@def create_version(obj, session, deleted=False):
if obj_changed is True:
break
+ log.info("%r relationship changed ? %r" % (obj, obj_changed))
+
if not obj_changed and not deleted:
return
- attr['version'] = obj.version
- hist = history_cls()
- for key, value in attr.items():
- setattr(hist, key, value)
- session.add(hist)
- obj.version += 1
+ hist_pk = (inspect(obj).identity, obj.version)
+ hist = session.info["history-objects"].get(hist_pk, None)
+ if hist is None:
+ hist = history_cls()
+ attr['version'] = obj.version
+ session.info["history-objects"].update({ hist_pk : hist })
+
+ for key, value in attr.items():
+ setattr(hist, key, value)
+
+ #session.add(hist)
+ #obj.version += 1
def versioned_session(session):
@event.listens_for(session, 'before_flush')
def before_flush(session, flush_context, instances):
- for obj in versioned_objects(session.dirty):
- create_version(obj, session)
- for obj in versioned_objects(session.deleted):
- create_version(obj, session, deleted=True)
+ versioned_session_before_flush_handler(session)
+
+ @event.listens_for(session, 'before_commit')
+ def before_commit(session):
+ versioned_session_before_commit_handler(session)
+
+def versioned_session_before_flush_handler(session):
+ if "history-objects" not in session.info:
+ log.info("history-objects added to session.info")
+ session.info["history-objects"] = dict()
+ for obj in versioned_objects(session.dirty):
+ create_version(obj, session)
+ for obj in versioned_objects(session.deleted):
+ create_version(obj, session, deleted=True)
+
+def versioned_session_before_commit_handler(session):
+ for hist in session.info["history-objects"].values():
+ session.add(hist)
+ vobj =
session.query(inspect(hist).class_.__versioning_cls__).get(hist.id)
+ assert vobj and hist.version == vobj.version, "vobj: %r hist:
%r" % (vobj, hist)
+ vobj.version = hist.version + 1
--
You received this message because you are subscribed to the Google
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to sqlalchemy+unsubscr...@googlegroups.com
<mailto:sqlalchemy+unsubscr...@googlegroups.com>.
To post to this group, send email to sqlalchemy@googlegroups.com
<mailto:sqlalchemy@googlegroups.com>.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
