[Zope-dev] zope.sqlalchemy

[Laurence Rowe]

Following discussions with Kapil, Christian and Martin I've developed 
zope.sqlalchemy. The aim is to provide a common base for transaction 
integration. It does not attempt to define any particular way to handle 
database configuration as there is not yet consensus on the best way to 
handle it.


I've uploaded it to zope svn and pypi. See 
http://pypi.python.org/pypi/zope.sqlalchemy


Currently it depends on a development version of SQLAlchemy. I hope to 
make a release following the 0.4.6 release of SQLAlchemy.


See pypi or the readme for details, but briefly usage is something like:

>>> engine = create_engine('sqlite:///')
>>> Session = scoped_session(sessionmaker(
... bind=engine, transactional=True, autoflush=True,
... extension=ZopeTransactionExtension()))
>>> session = Session()
>>> session.save(User(name='bob'))
>>> transaction.commit()

Any comments appreciated.

Laurence

___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy

[Andreas Jung]

--On 6. Mai 2008 02:26:58 +0100 Laurence Rowe <[EMAIL PROTECTED]> wrote:


Following discussions with Kapil, Christian and Martin I've developed
zope.sqlalchemy. The aim is to provide a common base for transaction
integration. It does not attempt to define any particular way to handle
database configuration as there is not yet consensus on the best way to
handle it.

I've uploaded it to zope svn and pypi. See
http://pypi.python.org/pypi/zope.sqlalchemy

Currently it depends on a development version of SQLAlchemy. I hope to
make a release following the 0.4.6 release of SQLAlchemy.

See pypi or the readme for details, but briefly usage is something like:

 >>> engine = create_engine('sqlite:///')
 >>> Session = scoped_session(sessionmaker(
 ... bind=engine, transactional=True, autoflush=True,
 ... extension=ZopeTransactionExtension()))
 >>> session = Session()
 >>> session.save(User(name='bob'))
 >>> transaction.commit()

Any comments appreciated.



Looks great (on the paper :-)). Trying to integrate it with z3c.sqlalchemy 
over the weekend. Thanks Laurence.


Andreas

pgpZKhiINSlh6.pgp
Description: PGP signature
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy

[Brian Sutherland]

On Tue, May 06, 2008 at 02:26:58AM +0100, Laurence Rowe wrote:
> Any comments appreciated.

It looks very, very good.

-- 
Brian Sutherland
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy

[Andreas Jung]

--On 6. Mai 2008 06:01:14 +0200 Andreas Jung <[EMAIL PROTECTED]> wrote:




--On 6. Mai 2008 02:26:58 +0100 Laurence Rowe <[EMAIL PROTECTED]> wrote:


Following discussions with Kapil, Christian and Martin I've developed
zope.sqlalchemy. The aim is to provide a common base for transaction
integration. It does not attempt to define any particular way to handle
database configuration as there is not yet consensus on the best way to
handle it.




Looks great (on the paper :-)). Trying to integrate it with
z3c.sqlalchemy over the weekend. Thanks Laurence.


I got a big stuck with the zope.sa integration in z3c.sa. Two tests of my 
testsuite fail (likely related to a session.save() operation):


Error in test testXXMapperGetMapper 
(z3c.sqlalchemy.tests.testSQLAlchemy.WrapperTests)

Traceback (most recent call last):
 File "/opt/python-2.4.4/lib/python2.4/unittest.py", line 260, in run
   testMethod()
 File 
"/Users/ajung/sandboxes/z3c.sqlalchemy/lib/python/z3c/sqlalchemy/tests/testSQLAlchemy.py", 
line 168, in testXXMapperGetMapper

   user = session.query(User).filter_by(firstname='foo')[0]
 File 
"/opt/python-2.4.4/lib/python2.4/site-packages/SQLAlchemy-0.4.5-py2.4.egg/sqlalchemy/orm/query.py", 
line 834, in __getitem__

   return list(self[item:item+1])[0]
 File 
"/opt/python-2.4.4/lib/python2.4/site-packages/SQLAlchemy-0.4.5-py2.4.egg/sqlalchemy/orm/query.py", 
line 925, in __iter__

   self.session._autoflush()
 File 
"/opt/python-2.4.4/lib/python2.4/site-packages/SQLAlchemy-0.4.5-py2.4.egg/sqlalchemy/orm/session.py", 
line 747, in _autoflush

   self.flush()
 File 
"/opt/python-2.4.4/lib/python2.4/site-packages/SQLAlchemy-0.4.5-py2.4.egg/sqlalchemy/orm/session.py", 
line 757, in flush

   self.uow.flush(self, objects)
 File 
"/opt/python-2.4.4/lib/python2.4/site-packages/SQLAlchemy-0.4.5-py2.4.egg/sqlalchemy/orm/unitofwork.py", 
line 236, in flush

   session.extension.after_flush(session, flush_context)
 File 
"/opt/python-2.4.4/lib/python2.4/site-packages/zope.sqlalchemy-0.1dev_r86482-py2.4.egg/zope/sqlalchemy/datamanager.py", 
line 187, in after_flush

   invalidate(session)
 File 
"/opt/python-2.4.4/lib/python2.4/site-packages/zope.sqlalchemy-0.1dev_r86482-py2.4.egg/zope/sqlalchemy/datamanager.py", 
line 170, in invalidate

   assert _SESSION_STATE[id(session)] is not STATUS_READONLY
KeyError: 32500656

The integration code follows your documentation;



And the tests are here:



Any idea?

Andreas



pgpDIQhCri8MD.pgp
Description: PGP signature
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] zope.sqlalchemy release

[Chris McDonough]

Tomorrow, I plan to:

- Merge the chrism-py3 branch of zope.sqlalchemy into its trunk.
  (http://svn.zope.org/zope.sqlalchemy/branches/chrism-py3/)
  to get Python 3 compatibility.

- Once the compat branch is merged, I'll make a 0.7 release of
  zope.sqlalchemy.

Any dissent?

- C



___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] zope.sqlalchemy, integration ideas

[Martijn Faassen]

Hi there,

Today I had a discussion with Jasper Spaans about how to go about 
improving megrok.rdb, Grok's relational database integration which aims 
to integrate Grok with SQLAlchemy. We started developing megrok.rdb at 
the Grokkerdam sprint a few weeks ago. We reread the discussion 
surrounding zope.sqlalchemy for ideas on how to go about integration and 
configuration. I think these discussions reach wider than just Grok's 
concerns. Note that I'm not proposing we fold any of these ideas into 
zope.sqlalchemy itself, which should remain as free of policy as 
possible; it could become (yet another) extension.


Let me run our current thinking by the list.

What would be nice in Zope applications (and we think would be good for 
Grok) would be per-instance database configuration. That is, we want to 
be able to install multiple instances of the same application and then 
configure each of them with a different database URN and it should all 
work, each talking to their own database.


Michael Bayer's suggestion involves the use of scoped sessions. He 
proposed the following code:


Session = scoped_session()

# start of request
engine = get_appropriate_engine()
Session(bind=engine)
try:
# do request
finally:
Session.remove()

Let's go through the steps. First it makes a scoped session object, it 
then configures it with the right engine at the start of the request (it 
can do this on a per-class level), and then at the end of the request it 
removes the Session again, which results in the actual session being closed.


Our get_appropriate_engine() would probably look the engine up as a 
local utility, as Laurence suggested. There is a bit of question about 
engine configuration, though.


If we want to support the use case of looking up the engine URL in a 
persistent datastore (for instance one URL per location), we have a 
question of ordering. We cannot do it too early; at the start of the 
transaction there isn't a ZODB yet to talk to so we can't look up a 
local utility. We can try doing it just in time:


_Session = scoped_session()

def Session(*args, **kw):
engine = get_appropriate_engine()
_Session.bind(bind=engine)
return _Session(*args, **kw)

Here get_appropriate_engine() could do a component.getUtility() and look 
up the engine for us, possibly in an application-local way. There's 
still the question of how this engine got configured in the first place. 
How does it know the database URL? How does the engine get created after 
the database URL is known (this might be quite late in the game; it 
could be stored in the ZODB). It then starts to look more and more 
attractive to do something similar like collective.lead's IDatabase 
utility, which can be stored persistently in the ZODB and has a 
getEngine() method which actually gets the engine (creating it if 
necessary).


If we use sqlalchemy.ext.declarative, we also need to make the 
declarative extension of SQLALchemy load up the tables at the right 
point in time.


We would also like a way to hook into matters and register some of our 
own tables and mappers manually. We figured perhaps the utility could 
fire an event that you can then write a handler for. This way there's 
less need to subclass the utility just to change some configuration 
(this is what collective.lead currently requires you to do). If a 
persistent local utility is in play, it shouldn't fire the configuration 
event during its own creation, as that would mean it'd only be fired 
once ever. We want to fire it just after engine creation.


I guess the database utility can remain quite simple. Its main tasks 
would be:


* allow access to the engine (creating it the first time)

* fire the event for additional configuration when the engine is first 
created


* maintain or somehow obtain the database URL. This could be retrieved 
from the ZODB if it's a local utility, or it could be hardcoded into a 
global utility, or it could be retrieved from some config file by a 
global utility.


We could have an expanded variety which also configures things using the 
SQLAlchemy declarative extension.


We still have the question of the 'remove()' bit in Michael's code. We 
looked at ScopedSession's remove() method, and it looks like it removes 
the session from the thread-local storage, and it actually closes the 
session.


Closing the session should be taken care of: zope.sqlalchemy's 
integration with Zope's transaction machinery will close the session. 
What about the registry cleanup that remove() appears to do? Is this 
currently being done by zope.sqlalchemy? Should it be?


Anyway, a whole lot of abstract talk. I still hope to get some feedback 
on this.


Regards,

Martijn

___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/lis

[Zope-dev] zope.sqlalchemy 0.6 release

[Laurence Rowe]

I've released zope.sqlalchemy 0.6 with the following changes:

* Implement should_retry for sqlalchemy.orm.exc.ConcurrentModificationError
  and serialization errors from PostgreSQL and Oracle.
  (Specify transaction>=1.1 to use this functionality.)

* Include license files.

* Add ``transaction_manager`` attribute to data managers for compliance with
  IDataManager interface.

Release available from http://pypi.python.org/pypi/zope.sqlalchemy

Laurence
___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy release

[Brian Sutherland]

On Mon, Dec 05, 2011 at 04:21:51PM -0500, Chris McDonough wrote:
> Tomorrow, I plan to:
> 
> - Merge the chrism-py3 branch of zope.sqlalchemy into its trunk.
>   (http://svn.zope.org/zope.sqlalchemy/branches/chrism-py3/)
>   to get Python 3 compatibility.
> 
> - Once the compat branch is merged, I'll make a 0.7 release of
>   zope.sqlalchemy.
> 
> Any dissent?

None from me!

This all looks like great work.

-- 
Brian Sutherland
___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy release

[Chris McDonough]

On Mon, 2011-12-05 at 16:21 -0500, Chris McDonough wrote:
> Tomorrow, I plan to:
> 
> - Merge the chrism-py3 branch of zope.sqlalchemy into its trunk.
>   (http://svn.zope.org/zope.sqlalchemy/branches/chrism-py3/)
>   to get Python 3 compatibility.
> 
> - Once the compat branch is merged, I'll make a 0.7 release of
>   zope.sqlalchemy.
> 
> Any dissent?

I've merged the chrism-py3 branch to trunk and I've made a 0.7 release
from the resulting trunk to PyPI.

- C


___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] zope.sqlalchemy+py3 test failures.

[Chris McDonough]

In case anyone is interested, I've made a stab at porting
zope.sqlalchemy to Python 3 at
http://svn.zope.org/zope.sqlalchemy/branches/chrism-py3/ .  Several
tests still fail.  I could use some help fixing them.

To run the test suite:

- Create a Python 3.2 virtualenv.

- Install nose into the virtualenv.

- Check out the "chrism-py3" transaction branch from
  svn://svn.zope.org/repos/main/transaction/branches/chrism-py3

- Install the "chrism-py3" transaction checkout into the virtualenv,
  e.g. $env32/bin/python setup.py develop.

- Install this package into the virtualenv eg. run $env32/bin/python
  setup develop.

- Run $env32/bin/python setup.py nosetests.

Currently 2 tests fail:

 [chrism@thinko zope.sqlalchemy]$ env32/bin/python setup.py test
running test
running egg_info
writing requirements to src/zope.sqlalchemy.egg-info/requires.txt
writing src/zope.sqlalchemy.egg-info/PKG-INFO
writing namespace_packages to
src/zope.sqlalchemy.egg-info/namespace_packages.txt
writing top-level names to src/zope.sqlalchemy.egg-info/top_level.txt
writing dependency_links to
src/zope.sqlalchemy.egg-info/dependency_links.txt
writing manifest file 'src/zope.sqlalchemy.egg-info/SOURCES.txt'
running build_ext
testTwoEngines (zope.sqlalchemy.tests.MultipleEngineTests) ... ok
testRetry (zope.sqlalchemy.tests.RetryTests) ... ERROR
testRetryThread (zope.sqlalchemy.tests.RetryTests) ... ERROR
testAbortAfterCommit (zope.sqlalchemy.tests.ZopeSQLAlchemyTests) ... ok
testAbortBeforeCommit (zope.sqlalchemy.tests.ZopeSQLAlchemyTests) ... ok
testBulkDelete (zope.sqlalchemy.tests.ZopeSQLAlchemyTests) ... ok
testBulkUpdate (zope.sqlalchemy.tests.ZopeSQLAlchemyTests) ... ok
testCommit (zope.sqlalchemy.tests.ZopeSQLAlchemyTests) ... ok
testCommitWithSavepoint (zope.sqlalchemy.tests.ZopeSQLAlchemyTests) ...
ok
testMarkUnknownSession (zope.sqlalchemy.tests.ZopeSQLAlchemyTests) ...
ok
testRelations (zope.sqlalchemy.tests.ZopeSQLAlchemyTests) ... ok
testRollbackAttributes (zope.sqlalchemy.tests.ZopeSQLAlchemyTests) ...
ok
testSavepoint (zope.sqlalchemy.tests.ZopeSQLAlchemyTests) ... ok
testSimplePopulation (zope.sqlalchemy.tests.ZopeSQLAlchemyTests) ... ok
testThread (zope.sqlalchemy.tests.ZopeSQLAlchemyTests) ... ok
testTransactionJoining (zope.sqlalchemy.tests.ZopeSQLAlchemyTests) ...
ok
testTwoPhase (zope.sqlalchemy.tests.ZopeSQLAlchemyTests) ... ok

==
ERROR: testRetry (zope.sqlalchemy.tests.RetryTests)
--
Traceback (most recent call last):
  File
"/home/chrism/projects/zope.sqlalchemy/env32/lib/python3.2/site-packages/SQLAlchemy-0.7.2-py3.2.egg/sqlalchemy/engine/base.py",
 line 1633, in _execute_context
context)
  File
"/home/chrism/projects/zope.sqlalchemy/env32/lib/python3.2/site-packages/SQLAlchemy-0.7.2-py3.2.egg/sqlalchemy/engine/default.py",
 line 327, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: no such table: test_users

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File
"/home/chrism/projects/zope.sqlalchemy/src/zope/sqlalchemy/tests.py",
line 513, in setUp
self.tm1.commit()
  File "/home/chrism/projects/transaction/transaction/_manager.py", line
89, in commit
return self.get().commit()
  File "/home/chrism/projects/transaction/transaction/_transaction.py",
line 342, in commit
reraise(t, v, tb)
  File "/home/chrism/projects/transaction/transaction/compat.py", line
60, in reraise
raise value
  File "/home/chrism/projects/transaction/transaction/_transaction.py",
line 333, in commit
self._commitResources()
  File "/home/chrism/projects/transaction/transaction/_transaction.py",
line 473, in _commitResources
reraise(t, v, tb)
  File "/home/chrism/projects/transaction/transaction/compat.py", line
60, in reraise
raise value
  File "/home/chrism/projects/transaction/transaction/_transaction.py",
line 445, in _commitResources
rm.tpc_begin(self)
  File
"/home/chrism/projects/zope.sqlalchemy/src/zope/sqlalchemy/datamanager.py", 
line 87, in tpc_begin
self.session.flush()
  File
"/home/chrism/projects/zope.sqlalchemy/env32/lib/python3.2/site-packages/SQLAlchemy-0.7.2-py3.2.egg/sqlalchemy/orm/session.py",
 line 1493, in flush
self._flush(objects)
  File
"/home/chrism/projects/zope.sqlalchemy/env32/lib/python3.2/site-packages/SQLAlchemy-0.7.2-py3.2.egg/sqlalchemy/orm/session.py",
 line 1562, in _flush
flush_context.execute()
  File
"/home/chrism/projects/zope.sqlalchemy/env32/lib/python3.2/site-packages/SQLAlchemy-0.7.2-py3.2.egg/sqlalchemy/orm/unitofwork.py",
 line 327, in execute
rec.execute(self)
  File
"/home/chrism/projects/zope.sqlalchemy/env32/lib/python3.2/site-packages/SQLAlchemy-0.7.2-py3.2.egg/sqlalchemy/orm/unitofwork.py",
 line 471, in execute
uow
  File
"/home/chrism/projects/zope.sqlalchemy/env32/lib/python3.2/site-packages/SQLAlchemy-0.7

[Zope-dev] zope.sqlalchemy and in-memory sqlite

[Malthe Borch]

With an in-memory engine, I seem to lose track of the tables after the 
first response.


Turn of events:

1. Request comes in
2. Create some tables
3. Add content and commit transaction
4. Query content
5. Return response

Now...

6. New request comes in
7. Query content

(OperationalError) no such table: mytable ...

With a disk-based database, no errors occur. The point of my little 
story line was to illustrate that this does not have anything to do with 
transactions being committed.


\malthe

___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy locks up transaction

[Brian Sutherland]

On Tue, Sep 16, 2008 at 02:01:07PM +0100, Laurence Rowe wrote:
> Brian Sutherland wrote:
> > Hi,
> > 
> > I've recently seen a situation where zope.sqlalchemy locked up the
> > transaction machinery. I'm not sure exactly what happened, but have
> > attached a failing test for at least one bug which may have caused it.
> > Hopefully it's self explanatory;)
> > 
> > If someone could help me solve this, that would be great!
> 
> Could you try this with latest trunk. I checked in a fix the other day 
> that may help.

I just checked in a fix, please feel free to comment on/revert it if
it's not up to standard:)

> 
> Laurence
> 
> ___
> Zope-Dev maillist  -  Zope-Dev@zope.org
> http://mail.zope.org/mailman/listinfo/zope-dev
> **  No cross posts or HTML encoding!  **
> (Related lists - 
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope )

-- 
Brian Sutherland
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy locks up transaction

[Laurence Rowe]

Brian Sutherland wrote:
> On Tue, Sep 16, 2008 at 02:01:07PM +0100, Laurence Rowe wrote:
>> Brian Sutherland wrote:
>>> Hi,
>>>
>>> I've recently seen a situation where zope.sqlalchemy locked up the
>>> transaction machinery. I'm not sure exactly what happened, but have
>>> attached a failing test for at least one bug which may have caused it.
>>> Hopefully it's self explanatory;)
>>>
>>> If someone could help me solve this, that would be great!
>> Could you try this with latest trunk. I checked in a fix the other day 
>> that may help.
> 
> I just checked in a fix, please feel free to comment on/revert it if
> it's not up to standard:)

Great, I'll take a look at it and do a release some time this week.

Laurence

___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy locks up transaction

[Dieter Maurer]

Brian Sutherland wrote at 2008-9-16 09:56 +0200:
>I've recently seen a situation where zope.sqlalchemy locked up the
>transaction machinery. I'm not sure exactly what happened, but have
>attached a failing test for at least one bug which may have caused it.
>Hopefully it's self explanatory;)
>
>If someone could help me solve this, that would be great!

We have seen similar problems and hope to understand/work around
them soon.



-- 
Dieter
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy locks up transaction

[Dieter Maurer]

Brian Sutherland wrote at 2008-9-17 12:33 +0200:
>On Tue, Sep 16, 2008 at 02:01:07PM +0100, Laurence Rowe wrote:
>> Brian Sutherland wrote:
>> > Hi,
>> > 
>> > I've recently seen a situation where zope.sqlalchemy locked up the
>> > transaction machinery. I'm not sure exactly what happened, but have
>> > attached a failing test for at least one bug which may have caused it.
>> > Hopefully it's self explanatory;)
>> > 
>> > If someone could help me solve this, that would be great!
>> 
>> Could you try this with latest trunk. I checked in a fix the other day 
>> that may help.
>
>I just checked in a fix, please feel free to comment on/revert it if
>it's not up to standard:)

It looks not yet right to clear the state in "tpc_vote" when
a two phase commit is used (which is now supported by "SQLAlchemy").

In addition, there may be a problem in case "session.close()" raises
an exception. Then, "_finish" would not be called.



-- 
Dieter
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy locks up transaction

[Brian Sutherland]

On Thu, Sep 18, 2008 at 07:08:56AM +0200, Dieter Maurer wrote:
> Brian Sutherland wrote at 2008-9-17 12:33 +0200:
> >On Tue, Sep 16, 2008 at 02:01:07PM +0100, Laurence Rowe wrote:
> >> Brian Sutherland wrote:
> >> > Hi,
> >> > 
> >> > I've recently seen a situation where zope.sqlalchemy locked up the
> >> > transaction machinery. I'm not sure exactly what happened, but have
> >> > attached a failing test for at least one bug which may have caused it.
> >> > Hopefully it's self explanatory;)
> >> > 
> >> > If someone could help me solve this, that would be great!
> >> 
> >> Could you try this with latest trunk. I checked in a fix the other day 
> >> that may help.
> >
> >I just checked in a fix, please feel free to comment on/revert it if
> >it's not up to standard:)
> 
> It looks not yet right to clear the state in "tpc_vote" when
> a two phase commit is used (which is now supported by "SQLAlchemy").

It looks to me that in a two phase commit, the tpc_vote which is called
is:

def tpc_vote(self, trans):
if self.tx is not None: # there may have been no work to do
self.tx.prepare()
self.state = 'voted'

is that what you mean by clearing state?

> In addition, there may be a problem in case "session.close()" raises
> an exception. Then, "_finish" would not be called.

I modified my patch, moving session.close() into _finish and actually
calling close() after the state is cleared. That way, the errors can
propagate, but the transaction will not get wedged.

-- 
Brian Sutherland
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy locks up transaction

[Dieter Maurer]

Brian Sutherland wrote at 2008-9-18 10:00 +0200:
> ...
>It looks to me that in a two phase commit, the tpc_vote which is called
>is:
>
>def tpc_vote(self, trans):
>if self.tx is not None: # there may have been no work to do
>self.tx.prepare()
>self.state = 'voted'
>
>is that what you mean by clearing state?

I have not recognized that there are two implementations for "tpc_vote" --
my criticims was against the wrong one.



-- 
Dieter
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy locks up transaction

[Hermann Himmelbauer]

Am Donnerstag 18 September 2008 07:08:56 schrieb Dieter Maurer:
> Brian Sutherland wrote at 2008-9-17 12:33 +0200:
> >On Tue, Sep 16, 2008 at 02:01:07PM +0100, Laurence Rowe wrote:
> >> Brian Sutherland wrote:
> >> > Hi,
> >> >
> >> > I've recently seen a situation where zope.sqlalchemy locked up the
> >> > transaction machinery. I'm not sure exactly what happened, but have
> >> > attached a failing test for at least one bug which may have caused it.
> >> > Hopefully it's self explanatory;)
> >> >
> >> > If someone could help me solve this, that would be great!
> >>
> >> Could you try this with latest trunk. I checked in a fix the other day
> >> that may help.
> >
> >I just checked in a fix, please feel free to comment on/revert it if
> >it's not up to standard:)
>
> It looks not yet right to clear the state in "tpc_vote" when
> a two phase commit is used (which is now supported by "SQLAlchemy").
>
> In addition, there may be a problem in case "session.close()" raises
> an exception. Then, "_finish" would not be called.

This may not be connected, but I just wanted to point out a problem with 
configurations with "autoflush=False". If I change some database object 
attributes in the session but do not call session.flush(), the data is not 
stored into the database, as no commit() is called, which would then flush 
out/commit the data.

So perhaps this is something to think about, or, at least something to be 
denoted in the documentation.

I circumvented the problem by setting autoflush=True and modifying my code so 
that it works (although it's quite complicated).

Best Regards,
Hermann

-- 
[EMAIL PROTECTED]
GPG key ID: 299893C7 (on keyservers)
FP: 0124 2584 8809 EF2A DBF9  4902 64B4 D16B 2998 93C7
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy locks up transaction

[Laurence Rowe]

2008/9/19 Hermann Himmelbauer <[EMAIL PROTECTED]>:
> Am Donnerstag 18 September 2008 07:08:56 schrieb Dieter Maurer:
>> Brian Sutherland wrote at 2008-9-17 12:33 +0200:
>> >On Tue, Sep 16, 2008 at 02:01:07PM +0100, Laurence Rowe wrote:
>> >> Brian Sutherland wrote:
>> >> > Hi,
>> >> >
>> >> > I've recently seen a situation where zope.sqlalchemy locked up the
>> >> > transaction machinery. I'm not sure exactly what happened, but have
>> >> > attached a failing test for at least one bug which may have caused it.
>> >> > Hopefully it's self explanatory;)
>> >> >
>> >> > If someone could help me solve this, that would be great!
>> >>
>> >> Could you try this with latest trunk. I checked in a fix the other day
>> >> that may help.
>> >
>> >I just checked in a fix, please feel free to comment on/revert it if
>> >it's not up to standard:)
>>
>> It looks not yet right to clear the state in "tpc_vote" when
>> a two phase commit is used (which is now supported by "SQLAlchemy").
>>
>> In addition, there may be a problem in case "session.close()" raises
>> an exception. Then, "_finish" would not be called.
>
> This may not be connected, but I just wanted to point out a problem with
> configurations with "autoflush=False". If I change some database object
> attributes in the session but do not call session.flush(), the data is not
> stored into the database, as no commit() is called, which would then flush
> out/commit the data.
>
> So perhaps this is something to think about, or, at least something to be
> denoted in the documentation.
>
> I circumvented the problem by setting autoflush=True and modifying my code so
> that it works (although it's quite complicated).

I don't think this is related to the above problem, but it may be a
bug in SQLAlchemy 0.4 whose behaviour I have replicated.

In 0.4 orm.session during a commit flush is called:

245 if self.autoflush:
246 self.session.flush()

whereas in 0.5 flush is called unconditionally so long as there is not
a flush in progress.

358 if not self.session._flushing:
359 self.session.flush()

The documentation for 0.4 appears to be misleading:

commit() serves two purposes; it issues a flush() unconditionally to
persist any remaining pending changes.


I think I should change zope.sqlalchemy to flush during it's tpc_begin
phase. This should only be done if a flush is not already in progress,
as otherwise sqlalchemy would raise an error. self.session._flushing
is only in 0.5 though, so maybe it is time to up the dependency. Or
maybe it is impossible for it to be in the flushing state at that
point anyway, as all user code will have completed.

Laurence
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy locks up transaction

[Hermann Himmelbauer]

Am Freitag 19 September 2008 11:59:55 schrieb Laurence Rowe:
> 2008/9/19 Hermann Himmelbauer <[EMAIL PROTECTED]>:
> > Am Donnerstag 18 September 2008 07:08:56 schrieb Dieter Maurer:
> >> Brian Sutherland wrote at 2008-9-17 12:33 +0200:
> >> >On Tue, Sep 16, 2008 at 02:01:07PM +0100, Laurence Rowe wrote:
> >> >> Brian Sutherland wrote:
> >> >> > Hi,
> >> >> >
> >> >> > I've recently seen a situation where zope.sqlalchemy locked up the
> >> >> > transaction machinery. I'm not sure exactly what happened, but have
> >> >> > attached a failing test for at least one bug which may have caused
> >> >> > it. Hopefully it's self explanatory;)
> >> >> >
> >> >> > If someone could help me solve this, that would be great!
> >> >>
> >> >> Could you try this with latest trunk. I checked in a fix the other
> >> >> day that may help.
> >> >
> >> >I just checked in a fix, please feel free to comment on/revert it if
> >> >it's not up to standard:)
> >>
> >> It looks not yet right to clear the state in "tpc_vote" when
> >> a two phase commit is used (which is now supported by "SQLAlchemy").
> >>
> >> In addition, there may be a problem in case "session.close()" raises
> >> an exception. Then, "_finish" would not be called.
> >
> > This may not be connected, but I just wanted to point out a problem with
> > configurations with "autoflush=False". If I change some database object
> > attributes in the session but do not call session.flush(), the data is
> > not stored into the database, as no commit() is called, which would then
> > flush out/commit the data.
> >
> > So perhaps this is something to think about, or, at least something to be
> > denoted in the documentation.
> >
> > I circumvented the problem by setting autoflush=True and modifying my
> > code so that it works (although it's quite complicated).
>
> I don't think this is related to the above problem, but it may be a
> bug in SQLAlchemy 0.4 whose behaviour I have replicated.
>
> In 0.4 orm.session during a commit flush is called:
>
> 245   if self.autoflush:
> 246   self.session.flush()
>
> whereas in 0.5 flush is called unconditionally so long as there is not
> a flush in progress.
>
> 358   if not self.session._flushing:
> 359   self.session.flush()
>
> The documentation for 0.4 appears to be misleading:
>
> commit() serves two purposes; it issues a flush() unconditionally to
> persist any remaining pending changes.

Ah, I see, that's really misleading.

> I think I should change zope.sqlalchemy to flush during it's tpc_begin
> phase. This should only be done if a flush is not already in progress,
> as otherwise sqlalchemy would raise an error. 

That's maybe a good idea.

> self.session._flushing 
> is only in 0.5 though, so maybe it is time to up the dependency. 

What do you mean with "up the dependency"? If you mean dropping support for SA 
0.4, please, *please* don't do that (0.5's autoflushing breaks things here 
and Database support for MaxDB/SAPDB is somehow flawed.). Due to many 
incompatibilites, I cannot switch to SA 0.5, so I'd really appreciate support 
for SA 0.4!

> Or 
> maybe it is impossible for it to be in the flushing state at that
> point anyway, as all user code will have completed.

If I understand it right, the problem does not occur in SA 0.5, right? If 
that's the case, maybe it is enough to outline the behaviour of SA 0.4 in the 
documentation and leave it as it is.

Best Regards,
Hermann

-- 
[EMAIL PROTECTED]
GPG key ID: 299893C7 (on keyservers)
FP: 0124 2584 8809 EF2A DBF9  4902 64B4 D16B 2998 93C7
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy locks up transaction

[Laurence Rowe]

2008/9/19 Laurence Rowe <[EMAIL PROTECTED]>:
> 2008/9/19 Hermann Himmelbauer <[EMAIL PROTECTED]>:
>> Am Donnerstag 18 September 2008 07:08:56 schrieb Dieter Maurer:
>>> Brian Sutherland wrote at 2008-9-17 12:33 +0200:
>>> >On Tue, Sep 16, 2008 at 02:01:07PM +0100, Laurence Rowe wrote:
>>> >> Brian Sutherland wrote:
>>> >> > Hi,
>>> >> >
>>> >> > I've recently seen a situation where zope.sqlalchemy locked up the
>>> >> > transaction machinery. I'm not sure exactly what happened, but have
>>> >> > attached a failing test for at least one bug which may have caused it.
>>> >> > Hopefully it's self explanatory;)
>>> >> >
>>> >> > If someone could help me solve this, that would be great!
>>> >>
>>> >> Could you try this with latest trunk. I checked in a fix the other day
>>> >> that may help.
>>> >
>>> >I just checked in a fix, please feel free to comment on/revert it if
>>> >it's not up to standard:)
>>>
>>> It looks not yet right to clear the state in "tpc_vote" when
>>> a two phase commit is used (which is now supported by "SQLAlchemy").
>>>
>>> In addition, there may be a problem in case "session.close()" raises
>>> an exception. Then, "_finish" would not be called.
>>
>> This may not be connected, but I just wanted to point out a problem with
>> configurations with "autoflush=False". If I change some database object
>> attributes in the session but do not call session.flush(), the data is not
>> stored into the database, as no commit() is called, which would then flush
>> out/commit the data.
>>
>> So perhaps this is something to think about, or, at least something to be
>> denoted in the documentation.
>>
>> I circumvented the problem by setting autoflush=True and modifying my code so
>> that it works (although it's quite complicated).
>
> I don't think this is related to the above problem, but it may be a
> bug in SQLAlchemy 0.4 whose behaviour I have replicated.
>
> In 0.4 orm.session during a commit flush is called:
>
> 245 if self.autoflush:
> 246 self.session.flush()
>
> whereas in 0.5 flush is called unconditionally so long as there is not
> a flush in progress.
>
> 358 if not self.session._flushing:
> 359 self.session.flush()
>
> The documentation for 0.4 appears to be misleading:
>
> commit() serves two purposes; it issues a flush() unconditionally to
> persist any remaining pending changes.
>
>
> I think I should change zope.sqlalchemy to flush during it's tpc_begin
> phase. This should only be done if a flush is not already in progress,
> as otherwise sqlalchemy would raise an error. self.session._flushing
> is only in 0.5 though, so maybe it is time to up the dependency. Or
> maybe it is impossible for it to be in the flushing state at that
> point anyway, as all user code will have completed.

Correction, this is my misunderstanding of sqlalchemy, as explained on irc:
 elro: that's when you do session.begin(autoflush=False)

So I should just make it an unconditional flush()

Laurence
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy+py3 test failures.

[Chris McDonough]

On Mon, 2011-09-26 at 21:47 -0400, Chris McDonough wrote:
> In case anyone is interested, I've made a stab at porting
> zope.sqlalchemy to Python 3 at
> http://svn.zope.org/zope.sqlalchemy/branches/chrism-py3/ .  Several
> tests still fail.  I could use some help fixing them.
> 
> To run the test suite:
> 
> - Create a Python 3.2 virtualenv.
> 
> - Install nose into the virtualenv.
> 
> - Check out the "chrism-py3" transaction branch from
>   svn://svn.zope.org/repos/main/transaction/branches/chrism-py3
> 
> - Install the "chrism-py3" transaction checkout into the virtualenv,
>   e.g. $env32/bin/python setup.py develop.
> 
> - Install this package into the virtualenv eg. run $env32/bin/python
>   setup develop.
> 
> - Run $env32/bin/python setup.py nosetests.
> 
> Currently 2 tests fail:
> 

Mike Merickel solved this...

This is because the setuptools (and nose) testrunners do not respect the
``test_suite`` stanza at the bottom of ``tests.py``; these shouldn't be
getting run unless the DSN has postgres in it, and they are getting run
regardless (under sqlite).

That brings up how to cope with running test suites that depend on
zope.testrunner features in a cross-platform way.  Currently the
bootstrap script in this package wont run under Py3.

- C


___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy+py3 test failures.

[Chris Withers]

On 27/09/2011 02:47, Chris McDonough wrote:
> In case anyone is interested, I've made a stab at porting
> zope.sqlalchemy to Python 3 at
> http://svn.zope.org/zope.sqlalchemy/branches/chrism-py3/ .  Several
> tests still fail.  I could use some help fixing them.

Have you tried running the tests by building the buildout instead?
(that's making a huge assumption that zc.buildout and zope.testing work 
on Py3, which I guess isn't the case...)

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
 - http://www.simplistix.co.uk
___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy+py3 test failures.

[Chris McDonough]

On Tue, 2011-09-27 at 06:43 +0100, Chris Withers wrote:
> On 27/09/2011 02:47, Chris McDonough wrote:
> > In case anyone is interested, I've made a stab at porting
> > zope.sqlalchemy to Python 3 at
> > http://svn.zope.org/zope.sqlalchemy/branches/chrism-py3/ .  Several
> > tests still fail.  I could use some help fixing them.
> 
> Have you tried running the tests by building the buildout instead?
> (that's making a huge assumption that zc.buildout and zope.testing work 
> on Py3, which I guess isn't the case...)

I'm not sure how to do it under Python 3, and I may be sufficiently
unmotivated to figure it out without some help. ;-)  The tests indeed
pass on python 2 using bin/test though, so if I could get the testrunner
running the tests under Py3, they would also at this point.

- C


___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy+py3 test failures.

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 09/27/2011 02:01 AM, Chris McDonough wrote:
> On Mon, 2011-09-26 at 21:47 -0400, Chris McDonough wrote:
>> In case anyone is interested, I've made a stab at porting 
>> zope.sqlalchemy to Python 3 at 
>> http://svn.zope.org/zope.sqlalchemy/branches/chrism-py3/ .
>> Several tests still fail.  I could use some help fixing them.
>> 
>> To run the test suite:
>> 
>> - Create a Python 3.2 virtualenv.
>> 
>> - Install nose into the virtualenv.
>> 
>> - Check out the "chrism-py3" transaction branch from 
>> svn://svn.zope.org/repos/main/transaction/branches/chrism-py3
>> 
>> - Install the "chrism-py3" transaction checkout into the
>> virtualenv, e.g. $env32/bin/python setup.py develop.
>> 
>> - Install this package into the virtualenv eg. run
>> $env32/bin/python setup develop.
>> 
>> - Run $env32/bin/python setup.py nosetests.
>> 
>> Currently 2 tests fail:
>> 
> 
> Mike Merickel solved this...
> 
> This is because the setuptools (and nose) testrunners do not respect
> the ``test_suite`` stanza at the bottom of ``tests.py``; these
> shouldn't be getting run unless the DSN has postgres in it, and they
> are getting run regardless (under sqlite).
> 
> That brings up how to cope with running test suites that depend on 
> zope.testrunner features in a cross-platform way.  Currently the 
> bootstrap script in this package wont run under Py3.

This bootstrap is from Jim's '2' branch of zc.buildout:

 
http://svn.zope.org/zc.buildout/branches/2/bootstrap/bootstrap.py?rev=121484&view=auto

It is designed to work with Py3k.


Tres.
- -- 
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk6B/IYACgkQ+gerLs4ltQ6XOwCeNuYNhjXbasb5Qfr729Howd5N
6sAAoIpnbXpvLMP340cyt3PghRjrah34
=933g
-END PGP SIGNATURE-
___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy+py3 test failures.

[Chris McDonough]

On Tue, 2011-09-27 at 12:40 -0400, Tres Seaver wrote:
> This bootstrap is from Jim's '2' branch of zc.buildout:
> 
>  
> http://svn.zope.org/zc.buildout/branches/2/bootstrap/bootstrap.py?rev=121484&view=auto
> 
> It is designed to work with Py3k.

I've replaced the bootstrap.py in the chrism-py3 branch of both the
transaction package and the zope.sqlalchemy packages with that one, and
everything looks good!

I think we can probably merge both of these branches to their respective
trunks and make releases.  I have the necessary powers to do it for
transaction; I'll try to track down elro to see if he's willing to do it
for zope.sqlalchemy.

- C


___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy+py3 test failures.

[Laurence Rowe]

On 29 September 2011 10:33, Chris McDonough  wrote:
> On Tue, 2011-09-27 at 12:40 -0400, Tres Seaver wrote:
>> This bootstrap is from Jim's '2' branch of zc.buildout:
>>
>>  http://svn.zope.org/zc.buildout/branches/2/bootstrap/bootstrap.py?rev=121484&view=auto
>>
>> It is designed to work with Py3k.
>
> I've replaced the bootstrap.py in the chrism-py3 branch of both the
> transaction package and the zope.sqlalchemy packages with that one, and
> everything looks good!
>
> I think we can probably merge both of these branches to their respective
> trunks and make releases.  I have the necessary powers to do it for
> transaction; I'll try to track down elro to see if he's willing to do it
> for zope.sqlalchemy.

I've added chrism as an owner. Before we make a final release I'd like
to revisit the savepoint release branches of transaction /
zope.sqlalchemy. I'll bring this up in another mail.

Laurence
___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy+py3 test failures.

[Chris Withers]

On 29/09/2011 15:22, Laurence Rowe wrote:
> On 29 September 2011 10:33, Chris McDonough  wrote:
>> On Tue, 2011-09-27 at 12:40 -0400, Tres Seaver wrote:
>>> This bootstrap is from Jim's '2' branch of zc.buildout:
>>>
>>>   
>>> http://svn.zope.org/zc.buildout/branches/2/bootstrap/bootstrap.py?rev=121484&view=auto
>>>
>>> It is designed to work with Py3k.
>>
>> I've replaced the bootstrap.py in the chrism-py3 branch of both the
>> transaction package and the zope.sqlalchemy packages with that one, and
>> everything looks good!
>>
>> I think we can probably merge both of these branches to their respective
>> trunks and make releases.  I have the necessary powers to do it for
>> transaction; I'll try to track down elro to see if he's willing to do it
>> for zope.sqlalchemy.
>
> I've added chrism as an owner. Before we make a final release I'd like
> to revisit the savepoint release branches of transaction /
> zope.sqlalchemy. I'll bring this up in another mail.

It'd also be great if we could get support for SA 0.7+'s new events 
system...

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
 - http://www.simplistix.co.uk
___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy+py3 test failures.

[Laurence Rowe]

On 29 September 2011 16:51, Chris Withers  wrote:
> It'd also be great if we could get support for SA 0.7+'s new events
> system...

I'm not convinced this will give any benefit. Currently we just require:

>>> Session = scoped_session(sessionmaker(bind=engine,
... extension=ZopeTransactionExtension()))

Instead we would require something like:

>>> Session = scoped_session(sessionmaker(bind=engine))
>>> ext = ZopeTransactionExtension()
>>> event.listen(Session, "after_attach", ext.after_attach)
>>> event.listen(Session, "after_begin", ext.after_begin)
>>> event.listen(Session, "after_flush", ext.after_flush)
>>> event.listen(Session, "after_bulk_update", ext.after_bulk_update)
>>> event.listen(Session, "after_bulk_delete", ext.after_bulk_delete)
>>> event.listen(Session, "before_commit", ext.before_commit)

Though this could become:

>>> Session = scoped_session(sessionmaker(bind=engine))
>>> ZopeTransactionExtension(Session)

I'm just not sure that it's worthwhile.

Laurence
___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy+py3 test failures.

[Chris McDonough]

On Thu, 2011-09-29 at 15:22 +0100, Laurence Rowe wrote:
> On 29 September 2011 10:33, Chris McDonough  wrote:
> > On Tue, 2011-09-27 at 12:40 -0400, Tres Seaver wrote:
> >> This bootstrap is from Jim's '2' branch of zc.buildout:
> >>
> >>  
> >> http://svn.zope.org/zc.buildout/branches/2/bootstrap/bootstrap.py?rev=121484&view=auto
> >>
> >> It is designed to work with Py3k.
> >
> > I've replaced the bootstrap.py in the chrism-py3 branch of both the
> > transaction package and the zope.sqlalchemy packages with that one, and
> > everything looks good!
> >
> > I think we can probably merge both of these branches to their respective
> > trunks and make releases.  I have the necessary powers to do it for
> > transaction; I'll try to track down elro to see if he's willing to do it
> > for zope.sqlalchemy.
> 
> I've added chrism as an owner. Before we make a final release I'd like
> to revisit the savepoint release branches of transaction /
> zope.sqlalchemy. I'll bring this up in another mail.

Sorry I haven't merged the transaction stuff to trunk yet.

While doing a review, I realize that one downside to doing this merge is
that it will kill Python 2.4 and 2.5 compatibility.  I think this is a
reasonable thing, given that Python 2.5 will no longer get any
maintenance as of today or so, and Python 2.4 hasn't gotten any for a
long time, and folks who need a 2.4/2.5 compat version can always use an
older version.

That said, before I go ahead and do this merge, is there anyone who
believes we should continue to support 2.4 and 2.5 in the transaction
package?  If so, for how long?

- C


___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy+py3 test failures.

[Chris McDonough]

On Sat, 2011-10-01 at 03:28 -0400, Chris McDonough wrote:
> On Thu, 2011-09-29 at 15:22 +0100, Laurence Rowe wrote:
> > I've added chrism as an owner. Before we make a final release I'd like
> > to revisit the savepoint release branches of transaction /
> > zope.sqlalchemy. I'll bring this up in another mail.
> 
> Sorry I haven't merged the transaction stuff to trunk yet.
> 
> While doing a review, I realize that one downside to doing this merge is
> that it will kill Python 2.4 and 2.5 compatibility.  I think this is a
> reasonable thing, given that Python 2.5 will no longer get any
> maintenance as of today or so, and Python 2.4 hasn't gotten any for a
> long time, and folks who need a 2.4/2.5 compat version can always use an
> older version.
> 
> That said, before I go ahead and do this merge, is there anyone who
> believes we should continue to support 2.4 and 2.5 in the transaction
> package?  If so, for how long?

Nobody responded to this, so I've merged the chrism-py3 branch into the
trunk.

Any word on the savepoint release branch featureset before I make a
release?

- C


___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] zope.sqlalchemy dependencies does not respect setup.py dev egg

[David Pratt]

Hi. I have been working with z3c.dobbin 0.4.1 which uses zope.sqlalchemy 
and z3c.saconfig. The last usable state was 4 days ago when changes were 
made to release of zope.sqlalchemy where dependencies were changed 
without changing the version.


Revision 88953 was made to zope.sqlalchemy but shows the same package 
version as 88952 (where SA dependencies were changed to no longer 
include beta0.5.2). beta0.5.2 was the requirement for z3c.dobbin and it 
will not work with beta0.5.3. Malthe is currently making changes to 
dobbin in the interim so it will eventually work again.


I thought by checking out revision 88952 and using it as a dev egg in my 
buildout, I could get back to work. Is this a bug in buildout since dev 
egg should take precedence regardless? I guess there were some problems 
with how this was handled. If trunk had been marked as 0.3dev it might 
have made the difference because both were 0.3 regardless of the change 
of state and dependencies. When I run the buildout the dev egg's 
requires.txt is below regardess of what is in setup.py.  Doesn't matter 
what I do my build fails. Any suggestions? Many thanks.


Regards,
David



requires.txt

setuptools
SQLAlchemy>=0.4.7,!=0.5.0beta1,!=0.5.0beta2
transaction
zope.interface

[test]
zope.testing
docutils


setup.py

install_requires=[
  'setuptools',
  'SQLAlchemy==0.5.0beta2', # or >=0.5b3
  'transaction',
  'zope.interface',
  ],
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope )

[Zope-dev] zope.sqlalchemy: Decoupling Zope3 and SQLAlchemy commit - how? Should I?

[Hermann Himmelbauer]

Hi,
In one of my Zope3 projects, I use lovely.remotetask to run specific tasks in 
background. These tasks do lengthy database operations via zope.sqlalchemy. 
Currently, lovely.remotetask seems to do a Zope3 commit() at the end of the 
task, which also does the SQLAlchemy commit.

However, some tasks consist of a loop over self-contained operations, 
therefore it would be handy to commit after each such operation. This way, if 
one of the operations fail (e.g. due to some database error, such as locking, 
refrential integrity violation etc.), not the whole task is rolled back but 
only one operation. Moreover, the probability of database locks is reduced.

But, how would I do that? First of all, I'd have to catch all database error, 
so that lovely.remotetask does not do a rollback on its own (which errors 
would that be?). Then I'd have to somehow commit - should I simply do a Zope3 
commit? Or should I somehow do a database commit via zope.sqlalchemy?

Best Regards,
Hermann

-- 
[EMAIL PROTECTED]
GPG key ID: 299893C7 (on keyservers)
FP: 0124 2584 8809 EF2A DBF9  4902 64B4 D16B 2998 93C7
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy: Decoupling Zope3 and SQLAlchemy commit - how? Should I?

[Laurence Rowe]

This depends on whether you have other resources involved in the
transaction or not. I would assume that you will only have the sql
database committing in the transaction, in which case it makes not a
blind bit of difference if you commit separately or not - just make
sure each individual task is a separate item in the queue and ensure
your queue processor can retry failures.

Laurence

2008/11/2 Hermann Himmelbauer <[EMAIL PROTECTED]>:
> Hi,
> In one of my Zope3 projects, I use lovely.remotetask to run specific tasks in
> background. These tasks do lengthy database operations via zope.sqlalchemy.
> Currently, lovely.remotetask seems to do a Zope3 commit() at the end of the
> task, which also does the SQLAlchemy commit.
>
> However, some tasks consist of a loop over self-contained operations,
> therefore it would be handy to commit after each such operation. This way, if
> one of the operations fail (e.g. due to some database error, such as locking,
> refrential integrity violation etc.), not the whole task is rolled back but
> only one operation. Moreover, the probability of database locks is reduced.
>
> But, how would I do that? First of all, I'd have to catch all database error,
> so that lovely.remotetask does not do a rollback on its own (which errors
> would that be?). Then I'd have to somehow commit - should I simply do a Zope3
> commit? Or should I somehow do a database commit via zope.sqlalchemy?
>
> Best Regards,
> Hermann
>
> --
> [EMAIL PROTECTED]
> GPG key ID: 299893C7 (on keyservers)
> FP: 0124 2584 8809 EF2A DBF9  4902 64B4 D16B 2998 93C7
>
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] zope.sqlalchemy: Decoupling Zope3 and SQLAlchemy commit - how? Should I?

[Brian Sutherland]

On Sun, Nov 02, 2008 at 09:37:56PM +0100, Hermann Himmelbauer wrote:
> Hi,
> In one of my Zope3 projects, I use lovely.remotetask to run specific tasks in 
> background. These tasks do lengthy database operations via zope.sqlalchemy. 
> Currently, lovely.remotetask seems to do a Zope3 commit() at the end of the 
> task, which also does the SQLAlchemy commit.
> 
> However, some tasks consist of a loop over self-contained operations, 
> therefore it would be handy to commit after each such operation. This way, if 
> one of the operations fail (e.g. due to some database error, such as locking, 
> refrential integrity violation etc.), not the whole task is rolled back but 
> only one operation. Moreover, the probability of database locks is reduced.

I've never used lovely.remotetask, but I would suggest trying:

* Have your "loop over self-contained operations" create more tasks.
* Move the loop to the application and create many tasks, one for
  each operation.

-- 
Brian Sutherland
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )