On Thu, Sep 4, 2014 at 5:53 PM, Michael Bayer <mike...@zzzcomputing.com>
wrote:
>
> > The code I say appears to expect the session_id to remain on detached
> states in certain situations is that the check in Session._attach checks
> not only the session_id but that the referenced session still exists in
> _sessions:
> > if state.session_id and \
> > state.session_id is not self.hash_key and \
> > state.session_id in _sessions:
> > raise sa_exc.InvalidRequestError(
> > "Object '%s' is already attached to session '%s' "
> > "(this is '%s')" % (orm_util.state_str(state),
> > state.session_id, self.hash_key))
> >
> > I am interested in this in the hopes that sheds light on the source of
> the intermittent failures.
>
> yeah that code is intended to detect when an object is added to a session
> that hasn't been removed from another.
>
>
So, is the intent of the 'state.session_id in _sessions" clause to handle
the situation where the session is collected while it contains instances?
> However, the implementation for session.close() has a direct trace that
> leads to the "session_id" set to None for all objects that are either in
> the identity map, or in the "_new" (pending) collection. There's no
> ambiguity there.
>
I agree, it appears to be pretty clear. However, on the application side I
have log messages indicating I successfully commit() and close() the
session before receiving an error saying an instance is still attached to
it:
2014-08-13 14:14:11 tuk1m1cm1 [debug] [DEBUG] db.py:57 - transaction 8710
started
2014-08-13 14:14:12 tuk1m1cm1 [debug] [DEBUG] db.py:143 - transaction 8710
committed
2014-08-13 14:14:12 tuk1m1cm1 [debug] [DEBUG] db.py:57 - transaction 8719
started
2014-08-13 14:14:13 tuk1m1cm1 [debug] [DEBUG] db.py:140 - transaction 8719
rolled back2014-08-13 14:14:13 tuk1m1cm1 [err] [ERROR] threadutils.py:98 -
Error executing asynchronous operation asynch-op <function execute at
0x5e54c08>(, {}) : Object '<Account at 0x5d49250>' is already attached to
session '8710' (this is '8719') at Traceback (most recent call last):...
"transaction" refers to a session, the integer is the session.hash_key.
> > In debugging this I have noticed that after the first session has been
> closed() and the initializer has returned the session remains in _sessions.
> However, if I call gc.collect() the session is removed, suggesting it just
> hasn't been fully cleaned up yet.
>
> that's not really accurate. There's a collection of all the Sessions in
> the weak referencing map _sessions, and Python's GC may not in fact remove
> those sessions in a deterministic way. But that has no bearing upon
> testing objects that have been removed from that session, and therefore
> have no session_id, if they belong to that session - their session_id has
> been cleared out.
>
>
I believe this is relevant because if the session were to be GC'ed the
"state.session_id in _sessions" would not evaluate to true and the attach
to the session I want would succeed.
>
> > Since it takes both the state referencing the session and the session
> still existing in _sessions, I can't help but wonder if this is a gc issue.
> Unfortunately it is not feasible to add gc.collect() calls in our
> production application as it introduces too much overhead.
>
> I will tell you how this *can* happen. If you have a Session, and you put
> objects in it, then you *don't* close the session; the session just is
> dereferenced, and is garbage collected at some point by the above mentioned
> weak dictionary. The objects within that session will *not* have anything
> done to their session_id in that case. If a new session is started up,
> and it happens to take over the same in-memory ID as the one that was just
> GC'ed, it will have the same hash key, and will then exist within
> _sessions. Then you move your objects to a third session; they will in
> fact have a session_id that is represented in _sessions, albeit against a
> different session than they started with, and you get the error.
>
> So if that's the case, then the issue here is the result of objects
> belonging to ad-hoc sessions that are not closed() explicitly, then those
> objects are shuttled along to another session, while busy thread mechanics
> in the background keep creating new sessions that occasionally use the same
> session identifier and produce this collision.
>
> There's ways to test for this, like assigning a unique counter to each
> Session within an event, like the begin() event, then using the
> before_attach event to verify that session_id is None, and if not, take a
> peek at that Session and its counter, compare it to something on the state.
>
>
I believe I have ruled this out with the "transaction XXX
started/committed" logging I added since I do not see hash keys being
reused.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> Visit this group at http://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at http://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
