Re: [Zope-dev] IRC discussion about testing

2009-08-25 Thread Stephan Richter 
On Tuesday 25 August 2009, Martijn Faassen wrote:
> > We can use such a list to extract more information from included
> > packages. For example, I have a script to extract all CHANGES.txt entries
> > in comparison to an older KGS version.
>
> Ah, nice use case. In fact I also have a script that parses CHANGES.txt
> entries and turns them into atom feeds, which could then be merged. I
> intended to use this to monitor the development of the KGS.

See zope.release for a full list of scripts that I originally developed for 
the KGS. I really wish that we are not loosing this sort of capabilities, 
since it would make the release process a lengthy task again.

Regards,
Stephan
-- 
Entrepreneur and Software Geek
Google me. "Zope Stephan Richter"
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] IRC discussion about testing

2009-08-25 Thread Martijn Faassen 
Stephan Richter wrote:
> On Wednesday 12 August 2009, Jim Fulton wrote:
>> I'm not sure why this all has to be computer readable.  What is the
>> use case for computer readable knowledge of whether a package is in
>> the kgs or just used to test it?  What is the need for computer
>> readable information about where the package is being developed.
> 
> We can use such a list to extract more information from included packages. 
> For 
> example, I have a script to extract all CHANGES.txt entries in comparison to 
> an older KGS version.

Ah, nice use case. In fact I also have a script that parses CHANGES.txt 
entries and turns them into atom feeds, which could then be merged. I 
intended to use this to monitor the development of the KGS.

Regards,

Martijn

___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] IRC discussion about testing

2009-08-25 Thread Martijn Faassen 
Jim Fulton wrote:
>> I do think we need a computer readable system that includes information
>> like this:
>>
>> * whether a project is in a KGS (such as for the ZTK)
>>
>> * whether a project is used to test a KGS (a package not in the ZTK but
>> used to test whether changes don't induce breakage, let's say,
>> grokcore.component)
>>
>> * the locked down version of the package.
>>
>> * where the *next* version of the locked down version is being
>> developed. Generally this is the SVN trunk, but could be a stable branch.
> 
> I'm not sure why this all has to be computer readable.  What is the
> use case for computer readable knowledge of whether a package is in
> the kgs or just used to test it?  

Making it computer readable to get those packages in the KGS is not that 
useful, I imagine, though making it so could potentially see tools being 
developed.

> What is the need for computer
> readable information about where the package is being developed.

That's a real usecase. There are tools that can already use this 
information, in particular mr.developer:

http://pypi.python.org/pypi/mr.developer

It's nice to be able to just get the relevant checkout of a package 
without having to hunt around trying to figure out where it is.

Regards,

Martijn

___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] IRC discussion about testing

2009-08-13 Thread Stephan Richter 
On Wednesday 12 August 2009, Jim Fulton wrote:
> I'm not sure why this all has to be computer readable.  What is the
> use case for computer readable knowledge of whether a package is in
> the kgs or just used to test it?  What is the need for computer
> readable information about where the package is being developed.

We can use such a list to extract more information from included packages. For 
example, I have a script to extract all CHANGES.txt entries in comparison to 
an older KGS version.

Thus, knowing the list of packages that we consider part of the KGS is very 
useful beyond testing, for example release management.

Regards,
Stephan

PS: I am still in Germany but will be back home late Sunday to read 
discussions more carefully.
-- 
Entrepreneur and Software Geek
Google me. "Zope Stephan Richter"
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] IRC discussion about testing

2009-08-12 Thread Wolfgang Schnerring 
* Jim Fulton  [2009-08-12 20:56]:
> This seems heavier than needed.  Also, if someone extends this,
> they're going to get an awful lot of sections that might have names
> that conflict with names in their buildout.  I do like the fact that
> the versions section is reusable. :)
>
> Here's an alternative:
>
>   [versions]
>   zope.foo = 1.2.3
>   zope.bar = 1.2.3
>   zope.app.baz = 1.2.3
>   grok.bar = 1.1.0
>   thirdparty.dependency = 4.4
>
>   [ztk]
>   projects = zope.foo zope.bar zope.app.baz
>   also-tested = grok.bar

Yup, that looks much better. As far as I'm concerned, let's use this.
(I'll leave it to Martijn to explain whether/which/why additional
information should be stored in the KGS in computer-readable form.)

I've just updated z3c.recipe.compattest to support exactly this:

>   [buildout]
>   parts = ztk-tests
>   extends = the-file-above
>
>   [ztk-tests]
>   recipe = z3c.recipe.compattest
>   include = ${ztk:projects} ${ztk:also-tested}

Would you give it a try?

Wolfgang

___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] IRC discussion about testing

2009-08-12 Thread Jim Fulton 
On Tue, Aug 11, 2009 at 7:36 PM, Jim Fulton wrote:
> On Tue, Aug 11, 2009 at 5:22 PM, Jim Fulton wrote:
> ...
>> - The versions specified in the controlled-packages.cfg don't seem to
>>  be honored except for the package being tested. For example,
>>  ZODB3-3.9.0b2 is used for test-kgs-ZODB3, but 3.9.0b5 is used for
>>  everything else.  Fortunately, it appears I can work around this
>>  using a buildout versions section.
>
> Except that there's a bug in buildout that causes extras to be dropped
> when versions are specified via a versions section.  This causes the
> extras to be omitted from the test scripts. :( Hopefully, I'll get a
> fixed buildout release out tomorrow.

Done

> The good news is that with a hacked buildout and a versions section, I
> can run tests for all of the packages in
> http://docs.zope.org/zopetoolkit/about/packages.html except for
> zope.ucol.  I'm getting test failures in 5 packages. I suspect that
> these are due to missing dependencies.

So, I created a KGS based on the ZTK and Fabio's recipe.  If you want
to try it, check out
svn://svn.zope.org/repos/main/zopetoolkit/branches/jim-kgs/kgs.

(I still think we want a simpler way to configure this, as proposed in
an earlier message today.)

Jim

-- 
Jim Fulton
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] IRC discussion about testing

2009-08-12 Thread Jim Fulton 
On Wed, Aug 12, 2009 at 7:07 AM, Wolfgang Schnerring wrote:
> * Jim Fulton  [2009-08-12 11:52]:
>> On Wed, Aug 12, 2009 at 2:31 AM, Wolfgang Schnerring wrote:
>>>* Jim Fulton  [2009-08-12 01:36]:
 In playing with this today, I'm inclined to think that it would be
 simpler to use a list of packages in an option to specify the packages
 to be tested and used a versions section to specify versions, rather
 than using a controlled-packages configuration file.
>>>
>>> This doesn't strike me as simpler to *use* (as I said earlier, I'd much
>>> prefer a *single* gesture of "use this KGS" to set up both the versions
>>> and the list of packages to run tests for, and I think it's worth
>>> spending effort to get there), but I'm not sure that's what you meant.
>>> What makes you prefer two files instead of one?
>>
>> I didn't say anything about 2 files.  I said I prefered a parts list
>> in a single option in combination with a standard buildout versions
>> section.
>
> Sorry for my misunderstanding. In fact, I'm not hung up about the number
> of files all that much, but rather I'm searching for a way not to
> duplicate information. And that seems rather diffcult, since you
> assert...
>
>> - We'll need the versions section to consume the KGS.  That is, given
>> a KGS, you'll often want to use the versions in other buildouts to
>> limit them to the known good configuration.
>
> ...while Martijn said here 
> 
> that the KGS will need to store more information about each package than
> a versions section can handle.

So there he said:

> I do think we need a computer readable system that includes information
> like this:
>
> * whether a project is in a KGS (such as for the ZTK)
>
> * whether a project is used to test a KGS (a package not in the ZTK but
> used to test whether changes don't induce breakage, let's say,
> grokcore.component)
>
> * the locked down version of the package.
>
> * where the *next* version of the locked down version is being
> developed. Generally this is the SVN trunk, but could be a stable branch.

I'm not sure why this all has to be computer readable.  What is the
use case for computer readable knowledge of whether a package is in
the kgs or just used to test it?  What is the need for computer
readable information about where the package is being developed.


>> - The parts section and versions section could be specified in a
>> single file, so the "gesture" for using them could be pretty simple.
>
> This seems to be the best we can do, given the above requirements.
> If I understand you correctly, that file would then look something like this:
>
> [versions]
> zope.foo = 1.2.3
> grok.bar = 1.1.0
> thirdparty.dependency = 4.4
>
> [zope.foo]
> tested = true
> kgs = ztk
> develop = http://svn.zope.org/repos/main/zope.foo/branches/1.2
>
> [grok.bar]
> tested = true
> develop = http://svn.zope.org/repos/main/grok.bar/trunk
>
> [thirdparty.dependency]
> tested = no
>
> z3c.recipe.compattest/kgs would learn to extract all sections from the
> above where tested=true. And zope.kgs/zope.release could then probably
> be retired (or am I missing something?).

This seems heavier than needed.  Also, if someone extends this,
they're going to get an awful lot of sections that might have names
that conflict with names in their buildout.  I do like the fact that
the versions section is reusable. :)

Here's an alternative:

  [versions]
  zope.foo = 1.2.3
  zope.bar = 1.2.3
  zope.app.baz = 1.2.3
  grok.bar = 1.1.0
  thirdparty.dependency = 4.4

  [ztk]
  projects = zope.foo zope.bar zope.app.baz
  also-tested = grok.bar

Now, if I want to test using this, I do something like:

  [buildout]
  parts = ztk-tests
  extends = the-file-above

  [ztk-tests]
  recipe = our.kgs.testing.recipe
  projects = ${ztk:projects} ${ztk:also-tested}

I think this is much cleaner and gives us:

- Easily reusable information, including versions information that can
  be used by any buildout.

- Information about the projects in the ztk

- Information about additional projects to be tested.

It doesn't give us machine-readable information about where the next
version of the project is being developed.  Why do we need that?

Also, with this, the recipe doesn't have to parse anything.

>> - I think a parts list in an option will be easier to control.  For
>> example, you will be able to use the standard buildout option
>> incrementing and decrementating machinery to modify it.
>
> I don't understand how a parts list could help here.

Now suppose we want to add another project to be tested, 'our.app':

  [buildout]
  parts = ztk-tests
  extends = the-file-above

  [ztk-tests]
  recipe = our.kgs.testing.recipe
  projects = ${ztk:projects} ${ztk:also-tested} our.app

But we don't want to bother testing zope.app.baz. We don't use it and
the tests take too long:

  [buildout]
  parts = ztk-tests
  extends = the-file-above

  [ztk-tests]
  recipe = our.kgs.testing.

Re: [Zope-dev] IRC discussion about testing

2009-08-12 Thread Wolfgang Schnerring 
* Jim Fulton  [2009-08-12 11:52]:
> On Wed, Aug 12, 2009 at 2:31 AM, Wolfgang Schnerring wrote:
>>* Jim Fulton  [2009-08-12 01:36]:
>>> In playing with this today, I'm inclined to think that it would be
>>> simpler to use a list of packages in an option to specify the packages
>>> to be tested and used a versions section to specify versions, rather
>>> than using a controlled-packages configuration file.
>>
>> This doesn't strike me as simpler to *use* (as I said earlier, I'd much
>> prefer a *single* gesture of "use this KGS" to set up both the versions
>> and the list of packages to run tests for, and I think it's worth
>> spending effort to get there), but I'm not sure that's what you meant.
>> What makes you prefer two files instead of one?
>
> I didn't say anything about 2 files.  I said I prefered a parts list
> in a single option in combination with a standard buildout versions
> section.

Sorry for my misunderstanding. In fact, I'm not hung up about the number
of files all that much, but rather I'm searching for a way not to
duplicate information. And that seems rather diffcult, since you
assert...

> - We'll need the versions section to consume the KGS.  That is, given
> a KGS, you'll often want to use the versions in other buildouts to
> limit them to the known good configuration.

...while Martijn said here 

that the KGS will need to store more information about each package than
a versions section can handle.

> - The parts section and versions section could be specified in a
> single file, so the "gesture" for using them could be pretty simple.

This seems to be the best we can do, given the above requirements.
If I understand you correctly, that file would then look something like this:

[versions]
zope.foo = 1.2.3
grok.bar = 1.1.0
thirdparty.dependency = 4.4

[zope.foo]
tested = true
kgs = ztk
develop = http://svn.zope.org/repos/main/zope.foo/branches/1.2

[grok.bar]
tested = true
develop = http://svn.zope.org/repos/main/grok.bar/trunk

[thirdparty.dependency]
tested = no

z3c.recipe.compattest/kgs would learn to extract all sections from the
above where tested=true. And zope.kgs/zope.release could then probably
be retired (or am I missing something?).

> - I think a parts list in an option will be easier to control.  For
> example, you will be able to use the standard buildout option
> incrementing and decrementating machinery to modify it.

I don't understand how a parts list could help here.

Wolfgang

___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] IRC discussion about testing

2009-08-12 Thread Jim Fulton 
On Wed, Aug 12, 2009 at 2:31 AM, Wolfgang Schnerring wrote:
> * Jim Fulton  [2009-08-12 01:36]:
>> On Tue, Aug 11, 2009 at 5:22 PM, Jim Fulton wrote:
>>> - The versions specified in the controlled-packages.cfg don't seem to
>>> be honored except for the package being tested. For example,
>>> ZODB3-3.9.0b2 is used for test-kgs-ZODB3, but 3.9.0b5 is used for
>>> everything else. Fortunately, it appears I can work around this
>>> using a buildout versions section.
>>
>> In playing with this today, I'm inclined to think that it would be
>> simpler to use a list of packages in an option to specify the packages
>> to be tested and used a versions section to specify versions, rather
>> than using a controlled-packages configuration file.
>
> This doesn't strike me as simpler to *use* (as I said earlier, I'd much
> prefer a *single* gesture of "use this KGS" to set up both the versions
> and the list of packages to run tests for, and I think it's worth
> spending effort to get there), but I'm not sure that's what you meant.
> What makes you prefer two files instead of one?

I didn't say anything about 2 files.  I said I prefered a parts list
in a single option in combination with a standard buildout versions
section.  There are a number of reasons for this:

- We'll need the versions section to consume the KGS.  That is, given
a KGS, you'll often want to use the versions in other buildouts to
limit them to the known good configuration.

- I think a parts list in an option will be easier to control.  For
example, you will be able to use the standard buildout option
incrementing and decrementating machinery to modify it.

- After using it for a day, I find the controlled-packages syntax to
be a bit heavy.  Of course, it didn't help that I still had to create
a versions list.

- The parts section and versions section could be specified in a
single file, so the "gesture" for using them could be pretty simple.

Jim

-- 
Jim Fulton
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] IRC discussion about testing

2009-08-11 Thread Wolfgang Schnerring 
* Jim Fulton  [2009-08-12 01:36]:
> On Tue, Aug 11, 2009 at 5:22 PM, Jim Fulton wrote:
>> - The versions specified in the controlled-packages.cfg don't seem to
>> be honored except for the package being tested. For example,
>> ZODB3-3.9.0b2 is used for test-kgs-ZODB3, but 3.9.0b5 is used for
>> everything else. Fortunately, it appears I can work around this
>> using a buildout versions section.
>
> In playing with this today, I'm inclined to think that it would be
> simpler to use a list of packages in an option to specify the packages
> to be tested and used a versions section to specify versions, rather
> than using a controlled-packages configuration file.

This doesn't strike me as simpler to *use* (as I said earlier, I'd much
prefer a *single* gesture of "use this KGS" to set up both the versions
and the list of packages to run tests for, and I think it's worth
spending effort to get there), but I'm not sure that's what you meant.
What makes you prefer two files instead of one?

I'm quite willing to either merge the z3c.recipe.kgs into compattest or
abandon the latter (or whatever) and to write a buildout extension to
pin the versions using a controlled-packages file, but I only want to do
that kind of work if we're reasonably certain that this is what we want.

Wolfgang

___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] IRC discussion about testing

2009-08-11 Thread Jim Fulton 
On Tue, Aug 11, 2009 at 5:22 PM, Jim Fulton wrote:
...
> - The versions specified in the controlled-packages.cfg don't seem to
>  be honored except for the package being tested. For example,
>  ZODB3-3.9.0b2 is used for test-kgs-ZODB3, but 3.9.0b5 is used for
>  everything else.  Fortunately, it appears I can work around this
>  using a buildout versions section.

Except that there's a bug in buildout that causes extras to be dropped
when versions are specified via a versions section.  This causes the
extras to be omitted from the test scripts. :( Hopefully, I'll get a
fixed buildout release out tomorrow.

The good news is that with a hacked buildout and a versions section, I
can run tests for all of the packages in
http://docs.zope.org/zopetoolkit/about/packages.html except for
zope.ucol.  I'm getting test failures in 5 packages. I suspect that
these are due to missing dependencies.

In playing with this today, I'm inclined to think that it would be
simpler to use a list of packages in an option to specify the packages
to be tested and used a versions section to specify versions, rather
than using a controlled-packages configuration file.

Jim

-- 
Jim Fulton
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] IRC discussion about testing

2009-08-11 Thread Jim Fulton 
2009/8/3 Fabio Tranchitella :
> Hi,

Hi,

I've added zope-dev. I hope you don't mind.

>
> * 2009-08-03 08:59, Christian Theune wrote:
>> Cool! Thanks for helping out! I should be online most of the time and
>> I'd be happy to answer questions.
>
> Tonight I spent some time working on this, and I wrote a mock-up of my idea
> for testing the KGS. I did not want to change anything in compattest, so I
> created a new recipe called z3c.recipe.kgs. Later on we can merge things
> back into compattest.
>
> You can get the recipe from here:
>
> svn://svn.zope.org/repos/main/z3c.recipe.kgs/trunk
>
> There are no tests nor documentation, because it is a mock-up and I did not
> want to spend time on writing documentation for something that maybe is not
> exactly what we need.
>
> Let's say that you are working on transaction and you want to test your
> changes against the KGS, you can simply add the following snippet to your
> buildout.cfg:
>
>  [test-kgs]
>  recipe = z3c.recipe.kgs
>  kgs = 
> http://svn.zope.org/*checkout*/zope.release/trunk/releases/controlled-packages.cfg
>
> You can now run buildout, and you will get ./bin/test-kgs- for
> each package in the KGS, as well as ./bin/test-kgs to run all the tests in
> all the packages in isolated environments.
>
> The KGS is rather big right now, so I suggest to try it with a subset of
> the packages using something like:
>
>  [test-kgs]
>  recipe = z3c.recipe.kgs
>  kgs = 
> http://svn.zope.org/*checkout*/zope.release/trunk/releases/controlled-packages.cfg
>  packages = zope.component
>zope.interface
>zope.event
>zope.configuration
>
> Now.. this simple recipe allows us to test a local change in a package we
> are developing on running the tests of all the packages that are part of a
> KGS. In fact, it reuses the KGS as defined in zope.release, without
> reinventing the wheel, and it is quite generic.
>
> Ideas? Comments? Is this the right direction?

I think this is the right direction. :)

A number of comments below:

- I just tried this out. Unfortunately, I'm getting lots of test
failures using Python 2.5.  That's not the recipe's fault. :)

- A bigger problem is that the runner of all tests eventually dies with
  too many open files on Mac OS X.  Here's the last part of the output:

...
Running test-kgs-zope.app.intid
Running test-kgs-zope.app.keyreference
Running test-kgs-zope.app.layers
Running test-kgs-zope.app.locales
1Running test-kgs-zope.app.localpermission
Running test-kgs-zope.app.locking
Traceback (most recent call last):
  File "bin/test-kgs", line 259, in 
  File "/private/tmp/kgs/zc.recipe.kgs/src/z3c/recipe/kgs/runner.py",
line 86, in main
  File "/private/tmp/kgs/zc.recipe.kgs/src/z3c/recipe/kgs/runner.py",
line 33, in start
  File "/usr/local/python/2.5.1/lib/python2.5/subprocess.py", line
593, in __init__
  File "/usr/local/python/2.5.1/lib/python2.5/subprocess.py", line
1002, in _execute_child
OSError: [Errno 24] Too many open files

- The packages from zope.release don't build for me on Windows because
of a dependency on M2Crypto from keas.kmi.  Not a big deal at this
point, since ultimately, we want to use a smaller set.

- The versions specified in the controlled-packages.cfg don't seem to
  be honored except for the package being tested. For example,
  ZODB3-3.9.0b2 is used for test-kgs-ZODB3, but 3.9.0b5 is used for
  everything else.  Fortunately, it appears I can work around this
  using a buildout versions section.

Thanks!

Jim

--
Jim Fulton
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )