Re: [ZODB-Dev] ZODB 3.9/3.8 incompatibility (was Re: Data.fs size grows non-stop)

2009-12-17 Thread Kapil Thangavelu
On Thu, Dec 10, 2009 at 8:12 PM, Marius Gedminas  wrote:

> On Thu, Dec 10, 2009 at 02:07:06PM -0500, Jim Fulton wrote:
> > On Thu, Dec 10, 2009 at 1:50 PM, Marius Gedminas 
> wrote:
> > > On Thu, Dec 10, 2009 at 12:41:11PM -0500, Jim Fulton wrote:
> > >> On Thu, Dec 10, 2009 at 11:54 AM, Marius Gedminas 
> wrote:
> > >> In 3.9 for FileStorage, if you give a starting tid that is toward the
> > >> end of the file,
> > >> it will scan backward, saving a lot of time.
> > >
> > > Ah, but how can I get a valid tid that is toward the end of the file?
> > > I assume that if I give a tid that doesn't exist, I'll get an error --
> > > which is the behavior oldstate() exhibits -- rather than the next
> > > existing transaction after that timestamp, or I could fudge by getting
> > > the current timestamp and subtracting 24 hours.
> >
> > No, the iterator starts at the first transactions who's tid is >= the
> > start tid.
> >
> > So, guess a time and use repr(ZODB.TimeStamp.TimeStamp(...))
>
> Cool!  Thank you,
>
> Marius Gedminas
> --
>
>
another way is to use the fstools.prev_txn starting at the end of the file
 to grab the newest transactions, the script i send to the list last week
uses it to poke a large zodb to just get the contents of the last 20
transactions without iterating through the whole thing or guessing a
timestamp.

https://mail.zope.org/pipermail/zodb-dev/attachments/20091207/531fc1e4/attachment.obj

cheers,

kapil
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9/3.8 incompatibility (was Re: Data.fs size grows non-stop)

2009-12-10 Thread Marius Gedminas
On Thu, Dec 10, 2009 at 02:07:06PM -0500, Jim Fulton wrote:
> On Thu, Dec 10, 2009 at 1:50 PM, Marius Gedminas  wrote:
> > On Thu, Dec 10, 2009 at 12:41:11PM -0500, Jim Fulton wrote:
> >> On Thu, Dec 10, 2009 at 11:54 AM, Marius Gedminas  wrote:
> >> In 3.9 for FileStorage, if you give a starting tid that is toward the
> >> end of the file,
> >> it will scan backward, saving a lot of time.
> >
> > Ah, but how can I get a valid tid that is toward the end of the file?
> > I assume that if I give a tid that doesn't exist, I'll get an error --
> > which is the behavior oldstate() exhibits -- rather than the next
> > existing transaction after that timestamp, or I could fudge by getting
> > the current timestamp and subtracting 24 hours.
> 
> No, the iterator starts at the first transactions who's tid is >= the
> start tid.
> 
> So, guess a time and use repr(ZODB.TimeStamp.TimeStamp(...))

Cool!  Thank you,

Marius Gedminas
-- 
This message has been ROT-13 encrypted twice for higher security.


signature.asc
Description: Digital signature
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9/3.8 incompatibility (was Re: Data.fs size grows non-stop)

2009-12-10 Thread Jim Fulton
On Thu, Dec 10, 2009 at 1:50 PM, Marius Gedminas  wrote:
> On Thu, Dec 10, 2009 at 12:41:11PM -0500, Jim Fulton wrote:
>> On Thu, Dec 10, 2009 at 11:54 AM, Marius Gedminas  wrote:
>> In 3.9 for FileStorage, if you give a starting tid that is toward the
>> end of the file,
>> it will scan backward, saving a lot of time.
>
> Ah, but how can I get a valid tid that is toward the end of the file?
> I assume that if I give a tid that doesn't exist, I'll get an error --
> which is the behavior oldstate() exhibits -- rather than the next
> existing transaction after that timestamp, or I could fudge by getting
> the current timestamp and subtracting 24 hours.

No, the iterator starts at the first transactions who's tid is >= the start tid.

So, guess a time and use repr(ZODB.TimeStamp.TimeStamp(...))

Jim

-- 
Jim Fulton
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9/3.8 incompatibility (was Re: Data.fs size grows non-stop)

2009-12-10 Thread Marius Gedminas
On Thu, Dec 10, 2009 at 12:41:11PM -0500, Jim Fulton wrote:
> On Thu, Dec 10, 2009 at 11:54 AM, Marius Gedminas  wrote:
> ...
> > Well, there is IStorage, and I think all storages (except for
> > FileStorage) implemented it faithfully
> 
> I guess that depends on what you mean by faithfully. MappingStorage
> and Demostorage
> in 3.8 inherited a null implementation from BaseStorage and it's
> arguable how faithful that
> implementation is.
> 
> > (oid,
> > version-that-got-removed-in-3.9, size=1).  FileStorage in 3.8 violated
> > the interface by renaming size=1 to length=1, but it's fixed in 3.9.
> 
> and in 3.8.

*checks*

FileStorage in 3.8.1 violated the interface, to be precise.

ZODB 3.8.1 is mandated by
http://download.zope.org/zope3.4/3.4.0/versions.cfg, which our
production app currently uses.

(Kinda offtopic: Is there a newer 3.4 KGS release?
http://download.zope.org/ tells me "Welcome" and nothing more.
http://download.zope.org/zope3.4/ tells me in great big joyful letters
at the top of the page "zope 3.4.0 Has Been Released!")

> > What I miss _a lot_ is a way to look at the last several transactions.
> > There's the iterator API, but it goes from oldest-to-latest, and when
> > you've got a large DB, that takes a while (85 seconds for a Data.fs with
> > 922737 transactions and 8 GB of data, with 100 MB/s read throughput).
> 
> In 3.9 for FileStorage, if you give a starting tid that is toward the
> end of the file,
> it will scan backward, saving a lot of time.

Ah, but how can I get a valid tid that is toward the end of the file?
I assume that if I give a tid that doesn't exist, I'll get an error --
which is the behavior oldstate() exhibits -- rather than the next
existing transaction after that timestamp, or I could fudge by getting
the current timestamp and subtracting 24 hours.

> I've attached an analysis script that illustrates iterating backward.

Thank you!

(Mutt is inconvenient: there's no way to look at attachments while you're
replying to a message.)

> Someday, someone should get around to adding an option to the file-storage
> iterator to go backward.

Marius Gedminas
-- 
Life begins when you can spend your spare time programming instead of
watching television.
-- Cal Keegan


signature.asc
Description: Digital signature
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9/3.8 incompatibility (was Re: Data.fs size grows non-stop)

2009-12-10 Thread Jim Fulton
On Thu, Dec 10, 2009 at 11:54 AM, Marius Gedminas  wrote:
...
> Well, there is IStorage, and I think all storages (except for
> FileStorage) implemented it faithfully

I guess that depends on what you mean by faithfully. MappingStorage
and Demostorage
in 3.8 inherited a null implementation from BaseStorage and it's
arguable how faithful that
implementation is.

> (oid,
> version-that-got-removed-in-3.9, size=1).  FileStorage in 3.8 violated
> the interface by renaming size=1 to length=1, but it's fixed in 3.9.

and in 3.8.

...

> What I miss _a lot_ is a way to look at the last several transactions.
> There's the iterator API, but it goes from oldest-to-latest, and when
> you've got a large DB, that takes a while (85 seconds for a Data.fs with
> 922737 transactions and 8 GB of data, with 100 MB/s read throughput).

In 3.9 for FileStorage, if you give a starting tid that is toward the
end of the file,
it will scan backward, saving a lot of time.

I've attached an analysis script that illustrates iterating backward.
Someday, someone should get around to adding an option to the file-storage
iterator to go backward.

Jim

-- 
Jim Fulton


transaction_rates.py
Description: Binary data
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9/3.8 incompatibility (was Re: Data.fs size grows non-stop)

2009-12-10 Thread Marius Gedminas
On Thu, Dec 10, 2009 at 10:08:36AM -0500, Jim Fulton wrote:
> On Thu, Dec 10, 2009 at 5:58 AM, Marius Gedminas  wrote:
> > On Wed, Dec 09, 2009 at 01:04:03PM -0500, Jim Fulton wrote:
> >> On Wed, Dec 9, 2009 at 12:06 PM, Marius Gedminas  wrote:
> >> ...
> >> > (Supporting both ZODB 3.8 and 3.9 is kinda tricky, but with some very
> >> > ugly hacks I managed.)
> >>
> >> This sounds like something that needs to be fixed. Can you share some of 
> >> the
> >> issues you ran into? (Or maybe file bugs reports.)
> >
> > Two issues only:
> >
> >  * In ZODB 3.8 PersistentDict and PersistentMapping are unrelated
> >    classes, so if I want to have adapters for both, I have to register
> >    them separately in ZCML.  In ZODB 3.9, PersistentDict is an alias
> >    for PersistentMapping, so I get a ZCML configuration conflict error
> >    if I have adapter directives for both.
> 
> OK, it's impractical to do anything about this.
> 
> >
> >    Solved by defining a decoy class for one of the adapters so they
> >    don't conflict.
> 
> Hm, I'm not sure I follow this. 

Code:

if PersistentMapping is PersistentDict:
# <...snip large comment explaining this...>
class DecoyPersistentDict(PersistentMapping):
"""Decoy to avoid ZCML errors while supporting both ZODB 3.8 and 
3.9."""
class PersistentDictState(PersistentMappingState):
"""Decoy to avoid ZCML errors while supporting both ZODB 3.8 and 
3.9."""
adapts(DecoyPersistentDict, dict, None)
else:
class PersistentDictState(PersistentMappingState):
"""Convenient access to a persistent dict's items."""
adapts(PersistentDict, dict, None)

ZCML:




> You could have used a different
> approach of providing the second adapter registration as an override.

Oooh, neat hack.  But I'm not sure I like it better than the one I
implemented.

> >  * In ZODB 3.8, the 'version' argument of ClientStorage.history (as well
> >    as other kinds of storages, I suppose) is mandatory.  In ZODB 3.9
> >    it's gone.

I went back and corrected my statement (originally it referred to
FileStorage.history, and I was thinking about ClientStorage in my
parenthetical statement), and then I of course forgot to remove the
now-obsolete parenthetical statement itself.

> It's mandatory in ClientStorage and optional in FileStorage. (MappingStorage 
> and
> DemoStorage don't have a history method in 3.8.
>
> I suggest we treat this as a bug in 3.8 and make the version argument
> to ClientStorage's history method optional in a 3.8 bug fix release.

+1

> >    Solved by peeking into the method signature with inspect.getargspec()
> >    and supplying a version only if it's needed.
> 
> Ick. :)

:)

> >> In particular, I think it should be a goal that it isn't too hard to
> >> write code that works with both ZODB 3.8 and 3.9.
> >
> > Normal code doesn't usually play with storage.history, I'd say, so I'm
> > not too peeved by having to do various contortions.
> 
> Zope 2 had (has?) a standard view for looking at an object's history
> that I found very useful. I wish there was one for Zope3.

Well, there is IStorage, and I think all storages (except for
FileStorage) implemented it faithfully (oid,
version-that-got-removed-in-3.9, size=1).  FileStorage in 3.8 violated
the interface by renaming size=1 to length=1, but it's fixed in 3.9.

I personally would like to be able to have size=None imply "unlimited,
give me as much as you've got"), but  works just as well
in practice.

What I miss _a lot_ is a way to look at the last several transactions.
There's the iterator API, but it goes from oldest-to-latest, and when
you've got a large DB, that takes a while (85 seconds for a Data.fs with
922737 transactions and 8 GB of data, with 100 MB/s read throughput).

> History can be very useful in my experience.

Oh yes.  This is why I stopped packing my production Data.fs'es and
first started using storage.history() with a couple of nice wrappers
from debugzope shell, and, later, convinced a coworker to write
ZODBBrowser.

Marius Gedminas
-- 
I used to think I was indecisive, but now I'm not so sure.


signature.asc
Description: Digital signature
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9/3.8 incompatibility (was Re: Data.fs size grows non-stop)

2009-12-10 Thread Jim Fulton
On Thu, Dec 10, 2009 at 5:58 AM, Marius Gedminas  wrote:
> On Wed, Dec 09, 2009 at 01:04:03PM -0500, Jim Fulton wrote:
>> On Wed, Dec 9, 2009 at 12:06 PM, Marius Gedminas  wrote:
>> ...
>> > (Supporting both ZODB 3.8 and 3.9 is kinda tricky, but with some very
>> > ugly hacks I managed.)
>>
>> This sounds like something that needs to be fixed. Can you share some of the
>> issues you ran into? (Or maybe file bugs reports.)
>
> Two issues only:
>
>  * In ZODB 3.8 PersistentDict and PersistentMapping are unrelated
>    classes, so if I want to have adapters for both, I have to register
>    them separately in ZCML.  In ZODB 3.9, PersistentDict is an alias
>    for PersistentMapping, so I get a ZCML configuration conflict error
>    if I have adapter directives for both.

OK, it's impractical to do anything about this.

>
>    Solved by defining a decoy class for one of the adapters so they
>    don't conflict.

Hm, I'm not sure I follow this.  You could have used a different
approach of providing the second adapter registration as an override.

>  * In ZODB 3.8, the 'version' argument of ClientStorage.history (as well
>    as other kinds of storages, I suppose) is mandatory.  In ZODB 3.9
>    it's gone.

It's mandatory in ClientStorage and optional in FileStorage. (MappingStorage and
DemoStorage don't have a history method in 3.8.

I suggest we treat this as a bug in 3.8 and make the version argument
to ClientStorage's history method optional in a 3.8 bug fix release.


>    Solved by peeking into the method signature with inspect.getargspec()
>    and supplying a version only if it's needed.

Ick. :)

>> In particular, I think it should be a goal that it isn't too hard to
>> write code that works with both ZODB 3.8 and 3.9.
>
> Normal code doesn't usually play with storage.history, I'd say, so I'm
> not too peeved by having to do various contortions.

Zope 2 had (has?) a standard view for looking at an object's history
that I found very useful. I wish there was one for Zope3.

History can be very useful in my experience.

Jim

-- 
Jim Fulton
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9/3.8 incompatibility (was Re: Data.fs size grows non-stop)

2009-12-10 Thread Marius Gedminas
On Thu, Dec 10, 2009 at 11:08:25AM +, Chris Withers wrote:
> Marius Gedminas wrote:
> >   * In ZODB 3.8, the 'version' argument of ClientStorage.history (as well
> > as other kinds of storages, I suppose) is mandatory.  In ZODB 3.9
> > it's gone.
> > 
> > Solved by peeking into the method signature with inspect.getargspec()
> > and supplying a version only if it's needed.
> 
> I believe this may be fixed with 3.9.4 is released.
> ...but it sounds like your code isn't ending up using the Z303 adapter, 
> which is why you're running into the problem.

What's Z303?

> How are you mixing versions? 3.8 on client and 3.9 on server or?

I'm not mixing versions at runtime; my intent is to write code that
works with either ZODB 3.8.x or 3.9.x.

You can see it here in all its glory:
http://bazaar.launchpad.net/~zodbbrowser-dev/zodbbrowser/trunk/annotate/head:/src/zodbbrowser/history.py#L30

http://bazaar.launchpad.net/~zodbbrowser-dev/zodbbrowser/trunk/annotate/head:/src/zodbbrowser/state.py#L115

> (sorry, I lost the previous thread and saw this as a new one when Jim 
> changed the subject line...)

This is a new topic, completely unrelated to the previous thread.

Marius Gedminas
-- 
I'm a shareware signature!  Send $2 if you use me, $10 for a manual.


signature.asc
Description: Digital signature
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9/3.8 incompatibility (was Re: Data.fs size grows non-stop)

2009-12-10 Thread Chris Withers
Marius Gedminas wrote:
>   * In ZODB 3.8, the 'version' argument of ClientStorage.history (as well
> as other kinds of storages, I suppose) is mandatory.  In ZODB 3.9
> it's gone.
> 
> Solved by peeking into the method signature with inspect.getargspec()
> and supplying a version only if it's needed.

I believe this may be fixed with 3.9.4 is released.
...but it sounds like your code isn't ending up using the Z303 adapter, 
which is why you're running into the problem.

How are you mixing versions? 3.8 on client and 3.9 on server or?
(sorry, I lost the previous thread and saw this as a new one when Jim 
changed the subject line...)

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
 - http://www.simplistix.co.uk
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9/3.8 incompatibility (was Re: Data.fs size grows non-stop)

2009-12-10 Thread Marius Gedminas
On Wed, Dec 09, 2009 at 01:04:03PM -0500, Jim Fulton wrote:
> On Wed, Dec 9, 2009 at 12:06 PM, Marius Gedminas  wrote:
> ...
> > (Supporting both ZODB 3.8 and 3.9 is kinda tricky, but with some very
> > ugly hacks I managed.)
> 
> This sounds like something that needs to be fixed. Can you share some of the
> issues you ran into? (Or maybe file bugs reports.)

Two issues only:

  * In ZODB 3.8 PersistentDict and PersistentMapping are unrelated
classes, so if I want to have adapters for both, I have to register
them separately in ZCML.  In ZODB 3.9, PersistentDict is an alias
for PersistentMapping, so I get a ZCML configuration conflict error
if I have adapter directives for both.

Solved by defining a decoy class for one of the adapters so they
don't conflict.

  * In ZODB 3.8, the 'version' argument of ClientStorage.history (as well
as other kinds of storages, I suppose) is mandatory.  In ZODB 3.9
it's gone.

Solved by peeking into the method signature with inspect.getargspec()
and supplying a version only if it's needed.

> In particular, I think it should be a goal that it isn't too hard to
> write code that works with both ZODB 3.8 and 3.9.

Normal code doesn't usually play with storage.history, I'd say, so I'm
not too peeved by having to do various contortions.

Marius Gedminas
-- 
Win32-API is wonderful : Full of things to wonder about...


signature.asc
Description: Digital signature
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] ZODB 3.9/3.8 incompatibility (was Re: Data.fs size grows non-stop)

2009-12-09 Thread Jim Fulton
On Wed, Dec 9, 2009 at 12:06 PM, Marius Gedminas  wrote:
...
> (Supporting both ZODB 3.8 and 3.9 is kinda tricky, but with some very
> ugly hacks I managed.)

This sounds like something that needs to be fixed. Can you share some of the
issues you ran into? (Or maybe file bugs reports.)

In particular, I think it should be a goal that it isn't too hard to
write code that
works with both ZODB 3.8 and 3.9.

Jim

-- 
Jim Fulton
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9

2009-04-12 Thread Jim Fulton

On Apr 12, 2009, at 4:27 AM, Dieter Maurer wrote:

> Hanno Schlichting wrote at 2009-4-11 14:43 +0200:
>> ...
>> ZODB 3.9 removed a bunch of deprecated API's. Look at
>> http://pypi.python.org/pypi/ZODB3/3.9.0a12#change-history to see how
>> much changed in this version.
>>
>> The main things were related to "Versions are no-longer supported."
>> which changed some low level API used in quite a number of places and
>> meant that some of the stuff in Products.OFSP couldn't possibly work
>> anymore.
>
> Hopefully, a ZODB 3.9 ZEO server is still able to speak with ZODB <  
> 3.9
> "ClientStorage" instances...


Yup. ZODB 3.9 storage servers support clients running ZODB 3.2 and  
later. Also, ZODB 3.9 clients can work with ZODB 3.8 storage servers.

Jim

--
Jim Fulton
Zope Corporation


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9

2009-04-12 Thread Dieter Maurer
Hanno Schlichting wrote at 2009-4-11 14:43 +0200:
> ...
>ZODB 3.9 removed a bunch of deprecated API's. Look at
>http://pypi.python.org/pypi/ZODB3/3.9.0a12#change-history to see how
>much changed in this version.
>
>The main things were related to "Versions are no-longer supported."
>which changed some low level API used in quite a number of places and
>meant that some of the stuff in Products.OFSP couldn't possibly work
>anymore.

Hopefully, a ZODB 3.9 ZEO server is still able to speak with ZODB < 3.9
"ClientStorage" instances...



-- 
Dieter
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9

2009-04-11 Thread Jim Fulton

Here's an update. I finally got around to installing the free MS  
express compiler so I can build on Windows with Python 2.6.  The ZODB  
tests run on my 32-bit VM, so that's encouraging. (People had reported  
the tests "crashing", which I assumed meant that the tests didn't even  
run.)  I'm getting the zeopack failure, which I'll investigate.  I  
didn't get the other 2 failures you reported.  I suspect they're  
intermittent.  I have a frequent problem with zeo_blob_cache.test,  
which is annoying, since I wrote it. :) It's testing a non-critical  
blob cleanup mechanism that happens in a thread and is subject to test  
race conditions.  So anyway, I think we're a lot closer than I feared.

Jim

On Apr 7, 2009, at 6:05 PM, Alan Runyan wrote:

>>> I was just reviewing what is pending for ZODB 3.9.
>>> https://bugs.launchpad.net/zodb/3.9/+bugs
>>>
>>> Looks like all of the bugs listed for 3.9 are fixed or fixes are  
>>> known
>>> and pending.
>>> Is a ZODB 3.9 beta around the corner?
>>>
>>> According to changes.txt the first alpha of 3.9 was cut almost 1/2  
>>> year ago
>>
>> ZODB + Python 2.6 on Windows is appearently an open issue.
>>
>> Andreas
>
> Where are the buildbot's/test runners for ZODB?  We just ran quick
> test and got the attached output.  I presume ZC has buildbot's for
> ZODB already running?
>  trunk-26amd64.txt>___
> For more information about ZODB, see the ZODB Wiki:
> http://www.zope.org/Wikis/ZODB/
>
> ZODB-Dev mailing list  -  ZODB-Dev@zope.org
> http://mail.zope.org/mailman/listinfo/zodb-dev

--
Jim Fulton
Zope Corporation


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9

2009-04-11 Thread Hanno Schlichting
Chris Withers wrote:
> Hanno Schlichting wrote:
>> Just be aware that ZODB 3.9 is not compatible with any stable Zope 2.x
>> release. It only works and is required for Zope 2.12. It can be made to
>> work with prior versions of Zope2 but that is a mild pain.
> 
> What are the problems with using ZODB 3.9 in Zope <2.12?

ZODB 3.9 removed a bunch of deprecated API's. Look at
http://pypi.python.org/pypi/ZODB3/3.9.0a12#change-history to see how
much changed in this version.

The main things were related to "Versions are no-longer supported."
which changed some low level API used in quite a number of places and
meant that some of the stuff in Products.OFSP couldn't possibly work
anymore. There were some smaller things as well, like ZopeUndo moving
into the Zope2 codebase and such.

I think someone reported to got the combination working, but I doubt
it's possible without editing the Zope2 source code, which isn't the
most maintainable solution.

Hanno

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9

2009-04-11 Thread Chris Withers
Hanno Schlichting wrote:
> Just be aware that ZODB 3.9 is not compatible with any stable Zope 2.x
> release. It only works and is required for Zope 2.12. It can be made to
> work with prior versions of Zope2 but that is a mild pain.

What are the problems with using ZODB 3.9 in Zope <2.12?

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
- http://www.simplistix.co.uk
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9

2009-04-08 Thread Jim Fulton

On Apr 8, 2009, at 11:02 AM, Alan Runyan wrote:

>> That would be great.  If we're talking Python 2.6, it might be best  
>> to
>> wait until someone gets the tests passing first.
>
> Ok.
>
> I believe it would look like (only on Windows):
>
> Python 2.4, 2.5, 2.6 on 32bit
> Python 2.6 on 64bit
>
> FWIW: Python 2.6.1 < there is a .dll loading regression for Windows.
> Fixed in trunk.

Please don't start running tests for Python 2.6 on windows until we  
get them passing.

...

>> You could email them to me, but please don't start emailing 2.6 tests
>> to me until we get them to pass. :)
>
> What about the zope-test mailing list?


Well, I don't read that regularly, but OK. :)

Jim

--
Jim Fulton
Zope Corporation


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9

2009-04-08 Thread Alan Runyan
> That would be great.  If we're talking Python 2.6, it might be best to
> wait until someone gets the tests passing first.

Ok.

I believe it would look like (only on Windows):

Python 2.4, 2.5, 2.6 on 32bit
Python 2.6 on 64bit

FWIW: Python 2.6.1 < there is a .dll loading regression for Windows.
Fixed in trunk.

>>  One problem is..
>> it takes ages for it the tests to run;
>
> About 45 minutes for me on a 32-bit VM with Python 2.5.

Sounds right.  42 minutes on the 64bit VM.

> You could email them to me, but please don't start emailing 2.6 tests
> to me until we get them to pass. :)

What about the zope-test mailing list?

We are aiming to have this done by mid next week.

cheers
alan
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9

2009-04-08 Thread Jim Fulton

On Apr 7, 2009, at 7:08 PM, Alan Runyan wrote:

> The tests were from ZODB trunk + Python 2.6 64bit on Windows.
>
> We could setup buildbot's for ZODB on Windows.

That would be great.  If we're talking Python 2.6, it might be best to  
wait until someone gets the tests passing first.

>  One problem is..
> it takes ages for it the tests to run;

About 45 minutes for me on a 32-bit VM with Python 2.5.


> so it would need to be on its
> own VM.
>
> I think we could run the tests nightly.. where would you want the  
> output?

You could email them to me, but please don't start emailing 2.6 tests  
to me until we get them to pass. :)

Jim

--
Jim Fulton
Zope Corporation


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9

2009-04-07 Thread Alan Runyan
The tests were from ZODB trunk + Python 2.6 64bit on Windows.

We could setup buildbot's for ZODB on Windows.  One problem is..
it takes ages for it the tests to run; so it would need to be on its
own VM.

I think we could run the tests nightly.. where would you want the output?

cheers
alan

p.s. no guarantees need to figure out how we could schedule them to
run at night as to not interfere with our other builds

On Tue, Apr 7, 2009 at 5:17 PM, Jim Fulton  wrote:
>
> On Apr 7, 2009, at 6:05 PM, Alan Runyan wrote:
>
 I was just reviewing what is pending for ZODB 3.9.
 https://bugs.launchpad.net/zodb/3.9/+bugs

 Looks like all of the bugs listed for 3.9 are fixed or fixes are
 known
 and pending.
 Is a ZODB 3.9 beta around the corner?

 According to changes.txt the first alpha of 3.9 was cut almost 1/2
 year ago
>>>
>>> ZODB + Python 2.6 on Windows is appearently an open issue.
>>>
>>> Andreas
>>
>> Where are the buildbot's/test runners for ZODB?  We just ran quick
>> test and got the attached output.
>
> What platform and Python version was this for?
>
>>  I presume ZC has buildbot's for
>> ZODB already running?
>
> No. Do you?  I typically run tests on Linux and Windows before
> committing, usually for Python 2.4 and 2.5. Lately, also on Python 2.6
> on Linux.  I would love someone to run some buildbots. I don't want to
> set them up myself. :)
>
> Jim
>
> --
> Jim Fulton
> Zope Corporation
>
>
> ___
> For more information about ZODB, see the ZODB Wiki:
> http://www.zope.org/Wikis/ZODB/
>
> ZODB-Dev mailing list  -  zodb-...@zope.org
> http://mail.zope.org/mailman/listinfo/zodb-dev
>



-- 
Alan Runyan
Enfold Systems, Inc.
http://www.enfoldsystems.com/
phone: +1.713.942.2377x111
fax: +1.832.201.8856
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9

2009-04-07 Thread Hanno Schlichting
Alan Runyan wrote:
> Is a ZODB 3.9 beta around the corner?

Just be aware that ZODB 3.9 is not compatible with any stable Zope 2.x
release. It only works and is required for Zope 2.12. It can be made to
work with prior versions of Zope2 but that is a mild pain.

Hanno

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9

2009-04-07 Thread Jim Fulton

On Apr 7, 2009, at 6:05 PM, Alan Runyan wrote:

>>> I was just reviewing what is pending for ZODB 3.9.
>>> https://bugs.launchpad.net/zodb/3.9/+bugs
>>>
>>> Looks like all of the bugs listed for 3.9 are fixed or fixes are  
>>> known
>>> and pending.
>>> Is a ZODB 3.9 beta around the corner?
>>>
>>> According to changes.txt the first alpha of 3.9 was cut almost 1/2  
>>> year ago
>>
>> ZODB + Python 2.6 on Windows is appearently an open issue.
>>
>> Andreas
>
> Where are the buildbot's/test runners for ZODB?  We just ran quick
> test and got the attached output.

What platform and Python version was this for?

>  I presume ZC has buildbot's for
> ZODB already running?

No. Do you?  I typically run tests on Linux and Windows before  
committing, usually for Python 2.4 and 2.5. Lately, also on Python 2.6  
on Linux.  I would love someone to run some buildbots. I don't want to  
set them up myself. :)

Jim

--
Jim Fulton
Zope Corporation


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9

2009-04-07 Thread Jim Fulton

On Apr 6, 2009, at 6:56 PM, Alan Runyan wrote:

> Hi.
>
> I was just reviewing what is pending for ZODB 3.9.
> https://bugs.launchpad.net/zodb/3.9/+bugs
>
> Looks like all of the bugs listed for 3.9 are fixed or fixes are known
> and pending.
> Is a ZODB 3.9 beta around the corner?

Yes, for some definition of around the corner.

The main blocker of a beta is integrating support for relstorage.

There are also issues with Python 2.6 on Windows, but that needn't  
block a beta.

Jim

--
Jim Fulton
Zope Corporation


___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZODB 3.9

2009-04-07 Thread Alan Runyan
>> I was just reviewing what is pending for ZODB 3.9.
>> https://bugs.launchpad.net/zodb/3.9/+bugs
>>
>> Looks like all of the bugs listed for 3.9 are fixed or fixes are known
>> and pending.
>> Is a ZODB 3.9 beta around the corner?
>>
>> According to changes.txt the first alpha of 3.9 was cut almost 1/2 year ago
>
> ZODB + Python 2.6 on Windows is appearently an open issue.
>
> Andreas

Where are the buildbot's/test runners for ZODB?  We just ran quick
test and got the attached output.  I presume ZC has buildbot's for
ZODB already running?
Tests with errors:
   checkConcurrentUpdates2StoragesMT 
(ZEO.tests.testConnection.FileStorageConnectionTests)

Tests with failures:
   checkConcurrentUpdates2StoragesMT 
(ZEO.tests.testConnection.FileStorageConnectionTests)
   d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\zeo_blob_cache.test
   d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\scripts\zeopack.test
Total: 3594 tests, 3 failures, 1 errors in 42 minutes 23.752 seconds.


Failure in test checkConcurrentUpdates2StoragesMT 
(ZEO.tests.testConnection.FileStorageConnectionTests)
Traceback (most recent call last):
  File "c:\Python26\lib\unittest.py", line 279, in run
testMethod()
  File 
"d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\InvalidationTests.py", line 
425, in checkConcurrentUpdates2StoragesMT
self.go(stop, cd, t1, t2, t3)
  File 
"d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\InvalidationTests.py", line 
323, in go
t.cleanup()
  File "d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\TestThread.py", line 
47, in run
self.testrun()
  File 
"d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\InvalidationTests.py", line 
54, in testrun
self._testrun()
  File 
"d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\InvalidationTests.py", line 
157, in _testrun
tree[key] = self.threadnum
  File "d:\temp\bot\zodb-trunk_maestro\build\src\ZODB\Connection.py", line 808, 
in setstate
self._setstate(obj)
  File "d:\temp\bot\zodb-trunk_maestro\build\src\ZODB\Connection.py", line 855, 
in _setstate
self._load_before_or_conflict(obj)
  File "d:\temp\bot\zodb-trunk_maestro\build\src\ZODB\Connection.py", line 890, 
in _load_before_or_conflict
if not self._setstate_noncurrent(obj):
  File "d:\temp\bot\zodb-trunk_maestro\build\src\ZODB\Connection.py", line 913, 
in _setstate_noncurrent
assert end is not None
AssertionError

 (9.689 s)


Error in test checkConcurrentUpdates2StoragesMT 
(ZEO.tests.testConnection.FileStorageConnectionTests)
Traceback (most recent call last):
  File "c:\Python26\lib\unittest.py", line 289, in run
self.tearDown()
  File "d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\ConnectionTests.py", 
line 163, in tearDown
os.unlink(path)  # sometimes this is just gonna fail
WindowsError: [Error 32] The process cannot access the file because it is being 
used by another process: '12-1.zec'



Failure in test 
d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\zeo_blob_cache.test
Traceback (most recent call last):
  File "c:\Python26\lib\unittest.py", line 279, in run
testMethod()
  File "c:\Python26\lib\doctest.py", line 2131, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for zeo_blob_cache.test
  File 
"d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\zeo_blob_cache.test", line 0

--
File "d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\tests\zeo_blob_cache.test", 
line 72, in zeo_blob_cache.test
Failed example:
cache_size('blobs') < 5000
Expected:
True
Got:
False


Failure in test 
d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\scripts\zeopack.test
Failed doctest test for zeopack.test
  File "d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\scripts\zeopack.test", 
line 0

--
File "d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\scripts\zeopack.test", line 
140, in zeopack.test
Failed example:
main(["-ufoo", "-d3", "-t1:30", "host1:8100:2"])
Expected:
ClientStorage(('host1', 8100), read_only=1, storage='2', wait=False)
is_connected True
pack(1237624200.0, wait=True)
close()
ClientStorage('foo', read_only=1, storage='1', wait=False)
is_connected True
pack(1237624200.0, wait=True)
close()
Got:
ClientStorage(('host1', 8100), read_only=1, storage='2', wait=False)
is_connected True
pack(1237617000.0, wait=True)
close()
ClientStorage('foo', read_only=1, storage='1', wait=False)
is_connected True
pack(1237617000.0, wait=True)
close()
--
File "d:\temp\bot\zodb-trunk_maestro\build\src\ZEO\scripts\zeopack.test", line 
158, in zeopack.test
Failed example:
main(["-d3", "-t1:30", "host1:8100:2"])
Expected:
ClientStorage(('host1', 8100), read_only=1, storage='2', wait=False)
is_connected False
sleep

[ZODB-Dev] ZODB 3.9

2009-04-06 Thread Alan Runyan
Hi.

I was just reviewing what is pending for ZODB 3.9.
https://bugs.launchpad.net/zodb/3.9/+bugs

Looks like all of the bugs listed for 3.9 are fixed or fixes are known
and pending.
Is a ZODB 3.9 beta around the corner?

According to changes.txt the first alpha of 3.9 was cut almost 1/2 year ago

cheers
alan
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev