Re: Testlistener Events in JUnit Slows Down Forked Tests in 1.8.x

[Stefan Bodewig]

On 2010-08-10, Danny Yates wrote:

> Sorry, I didn't make myself clear. I was actually thinking of using a read
> timeout so that the read call returns periodically and you can test
> interrupted(), but on reflection, you can only do that with sockets.

I think you will be able to do that with NIO2 (at worst emulate it with
async-IO) which is many years away from being usable in Ant's core.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

Re: Testlistener Events in JUnit Slows Down Forked Tests in 1.8.x

[Danny Yates]

Sorry, I didn't make myself clear. I was actually thinking of using a read
timeout so that the read call returns periodically and you can test
interrupted(), but on reflection, you can only do that with sockets.

I'll get my coat... ;-)


On 10 August 2010 20:45, Stefan Bodewig  wrote:

> On 2010-08-10, Danny Yates wrote:
>
> > Why does the StreamPumper poll like that?
>
> So it can be interrupted.
>
> > Surely it should just block reading the InputStream?
>
> It used to do just that.
>
> On Windows if a process spawns children (grandchildren of Ant in this
> case) they inherit the streams of their parent.  In cases where a
> process forked by Ant spawns new processes and then finishs Ant would
> wait for the grandchildren which is not desired.  Instead we want Ant to
> go on once the forked process is done.
>
> > To deal with InterruptedException, you can just do a timed read and
> > test the interrupted flag before looping back to the blocking read.
>
> I'm not sure I follow you here.  Are you talking about NIO's Selector
> with "timed read"?  When Ant was changed to use polling we still lived
> in Java 1.3 API land and so never looked at NIO options.  I'm afraid the
> InputStreams provided by the Process class won't provide
> SelectableChannels, though.
>
> Stefan
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
> For additional commands, e-mail: dev-h...@ant.apache.org
>
>

Re: Testlistener Events in JUnit Slows Down Forked Tests in 1.8.x

[Stefan Bodewig]

On 2010-08-10, Danny Yates wrote:

> Why does the StreamPumper poll like that?

So it can be interrupted.

> Surely it should just block reading the InputStream?

It used to do just that.

On Windows if a process spawns children (grandchildren of Ant in this
case) they inherit the streams of their parent.  In cases where a
process forked by Ant spawns new processes and then finishs Ant would
wait for the grandchildren which is not desired.  Instead we want Ant to
go on once the forked process is done.

> To deal with InterruptedException, you can just do a timed read and
> test the interrupted flag before looping back to the blocking read.

I'm not sure I follow you here.  Are you talking about NIO's Selector
with "timed read"?  When Ant was changed to use polling we still lived
in Java 1.3 API land and so never looked at NIO options.  I'm afraid the
InputStreams provided by the Process class won't provide
SelectableChannels, though.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

Re: Testlistener Events in JUnit Slows Down Forked Tests in 1.8.x

[Danny Yates]

Why does the StreamPumper poll like that? Surely it should just block
reading the InputStream?

To deal with InterruptedException, you can just do a timed read and test the
interrupted flag before looping back to the blocking read.

Danny.


On 10 August 2010 20:00, Stefan Bodewig  wrote:

> On 2010-08-10, Jesse Glick wrote:
>
> > On 08/09/2010 03:42 AM, Stefan Bodewig wrote:
> >> I plan to add an disableTestListenerEvents
>
> > First I would like to hear an explanation of why printing a few lines
> > to System.out causes such a big performance hit and why this cannot be
> > fixed.
>
> "such a big" is just the accumulation of many small hits.
>
> A test that takes a few miliseconds without the lines to System.out may
> be unfortunate enough to always print lines just after the StreamPumper
> has entered the sleep call and then must wait until StreamPumper wakes
> up before it can proceed.  There is one such line before and after each
> test method and each stands a chance to hit the POLL_TIMEOUT which is
> 100ms.
>
> In the very simple test case Laura provided all test together finish in
> less than 10ms and just one of them hitting the sleep is enough to make
> the entire runtime 10 times as much.
>
> In reality each such line is pretty likely to be written while the
> pumper is sleeping, each line is pretty likely to be delayed by up to
> 100ms, even though it will be a lot shorter than that in most cases.  If
> you have many test, the times sum up.  This isn't anything you notice in
> a testsuite where individual tests take long - like Ant's own testsuite
> whose runtime is dominated by test methods that take many seconds - but
> in setups with very many very small test.
>
> > Does the problem only occur on Windows?
>
> In 1.8.0, yes.  In theory the fix for Bug 48746 has ported the effect to
> any OS with Ant 1.8.1 but I've been unable to see the effect on Linux -
> at least with Laura's small testcase.  I must admit I haven't tried it
> again on any OS but Windows after I figured out what was going on.
>
> > And can it be avoided by just removing calls to flush(),
>
> No, that doesn't have any effect.  Thinking of it, buffering on the
> testrunner side would help but at the same time make the lines unusable
> for the test listener events that must happen in step with the test to
> be useful.
>
> > possibly using System.err?
>
> Uses a different StreamPumper instance but would suffer from the same
> effect.
>
> >> We could add a YAMP (yet another magic property) ...
>
> > This would be much preferable IMHO. The property would just be set by
> > any container which wants to listen to  progress.  (An
> > attribute in the build script is useless for this purpose, even if the
> > default is on - since the user might turn it off for speed of builds
> > in other environments!)
>
> You mean the magic property would enable the events even if the user
> wants to turn them off?  Or do you want to turn events off by default
> and enable them based on the magic property?  The later would break
> backwards compatibility.
>
> Stefan
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
> For additional commands, e-mail: dev-h...@ant.apache.org
>
>

Re: Testlistener Events in JUnit Slows Down Forked Tests in 1.8.x

[Stefan Bodewig]

On 2010-08-10, Jesse Glick wrote:

> On 08/09/2010 03:42 AM, Stefan Bodewig wrote:
>> I plan to add an disableTestListenerEvents

> First I would like to hear an explanation of why printing a few lines
> to System.out causes such a big performance hit and why this cannot be
> fixed.

"such a big" is just the accumulation of many small hits.

A test that takes a few miliseconds without the lines to System.out may
be unfortunate enough to always print lines just after the StreamPumper
has entered the sleep call and then must wait until StreamPumper wakes
up before it can proceed.  There is one such line before and after each
test method and each stands a chance to hit the POLL_TIMEOUT which is
100ms.

In the very simple test case Laura provided all test together finish in
less than 10ms and just one of them hitting the sleep is enough to make
the entire runtime 10 times as much.

In reality each such line is pretty likely to be written while the
pumper is sleeping, each line is pretty likely to be delayed by up to
100ms, even though it will be a lot shorter than that in most cases.  If
you have many test, the times sum up.  This isn't anything you notice in
a testsuite where individual tests take long - like Ant's own testsuite
whose runtime is dominated by test methods that take many seconds - but
in setups with very many very small test.

> Does the problem only occur on Windows?

In 1.8.0, yes.  In theory the fix for Bug 48746 has ported the effect to
any OS with Ant 1.8.1 but I've been unable to see the effect on Linux -
at least with Laura's small testcase.  I must admit I haven't tried it
again on any OS but Windows after I figured out what was going on.

> And can it be avoided by just removing calls to flush(),

No, that doesn't have any effect.  Thinking of it, buffering on the
testrunner side would help but at the same time make the lines unusable
for the test listener events that must happen in step with the test to
be useful.

> possibly using System.err?

Uses a different StreamPumper instance but would suffer from the same
effect.

>> We could add a YAMP (yet another magic property) ...

> This would be much preferable IMHO. The property would just be set by
> any container which wants to listen to  progress.  (An
> attribute in the build script is useless for this purpose, even if the
> default is on - since the user might turn it off for speed of builds
> in other environments!)

You mean the magic property would enable the events even if the user
wants to turn them off?  Or do you want to turn events off by default
and enable them based on the magic property?  The later would break
backwards compatibility.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

Re: Testlistener Events in JUnit Slows Down Forked Tests in 1.8.x

[Jesse Glick]

On 08/09/2010 03:42 AM, Stefan Bodewig wrote:

I plan to add an disableTestListenerEvents


First I would like to hear an explanation of why printing a few lines to System.out causes such a big performance hit and why this cannot be fixed. What is waiting for 
what to finish? Does the problem only occur on Windows? And can it be avoided by just removing calls to flush(), possibly using System.err?



We could add a YAMP (yet another magic property) ...


This would be much preferable IMHO. The property would just be set by any container which wants to listen to  progress. (An attribute in the build script is 
useless for this purpose, even if the default is on - since the user might turn it off for speed of builds in other environments!)



-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

Easyant 0.8 released !

[Jean-Louis Boudart]

The EasyAnt project is pleased to announce its 0.8 version.

Easyant is a build system, that is based on Apache Ant and Apache Ivy.

Our goals are :

   - to leverage popularity and flexibility of Ant.
   - to integrate Apache Ivy, such that the build system combines a
   ready-to-use dependency manager.
   - to simplify standard build types, such as building web applications,
   JARs etc, by providing ready to use builds.
   - to provide conventions and guidelines.
   - to make plugging-in of fresh functionalities easy as writing simple Ant
   scripts as Easyant plugins.

To still remain adaptable,

   - Though Easyant comes with a lot of conventions, we never lock you in.
   - Easyant allows you to easily extend existing modules or create and use
   your own modules.
   - Easyant makes migration from Ant very simple. Your legacy Ant scripts
   could still be leveraged with Easyant.

Changelog 0.8 :

Core:

   - Feature #132 : Switch to Ivy 2.2.0-rc1
   - Feature #117 : Switch to ant 1.8.1
   - Feature #136 : Introduce bindtarget task

   This tasks allow to change the mapping between target and phases.
   - Feature #137 : Introduce findclasspath
   task
   This task is designed to locate the right version of a SDK (for groovy /
   scala, but maybe for others SDK like android etc...).
   - Feature #138 : Introduce
   registerartifact task
   This task is in charge to register a given artifact to ivy context.
   In other words, this task is responsible of generating the right elements
   in the publicated module.ivy file.
   - Feature #70 : Introduce a menu generator
   task
   This task is used by documentation plugins (like xooki) to generate menu
   entries
   - Feature #116 : Introduce CheckResolver
   task to enforce plugin stability
   - Feature #124 : Introduce a ant task to
   run an easyant build process
   - Feature #100 : support for extends=info
   in ivy module parser
   - Feature #101 : publish merged version of
   Ivy descriptor when using module inheritence ( tag in a module.ivy
   file)
   - Feature #119 : EasyAnt's launch scripts
   now support alternative ant distribution
   - Feature #122 : Simplify usage of project
   ivy settings (if exists use ivysettings.xml at the root level of the
   project)
   - Many bug fixes on core and documentation (plugin documentation now
   contains examples)
   - Simplify the build process of easyant core itself

Plugins :

   - Feature #94 : Introduce package-test-jar
   plugin
   - Feature #113 : Introduce
   maven-publication plugin
   - Feature #127 : Introduce distribution
   plugin
   - Feature #115 : Add support for
   integration tests
   - Feature #123 : Support filterset in
   resources-std plugin
   - Feature #98 : Support for
   pre-module-phases in multi module
   - Feature #128 : Add a new target on
   skeleton plugin to choose a skeleton from a list
   - Feature #111 : implement xooki
   menugenerator
   - Feature #92 : eadoc plugin should use ant
   uptodate check to improve performance
   - Feature #93 : add resource collection
   support to junit plugin
   - Feature #109 : Homogenize properties
   referencing the main class
   - Several bugfixes on
  - skeleton
  - mannifest
  - antunit
  - ivy-provisionning

You can have further details on EasyAnt 0.8 in the release
notes
.

Issues should be reported to: http://easyant.org/projects/easyant

Retrieve sources from the 0.8 release files at:
http://svn.easyant.org/tags/0.8/
Or download the 0.8 release files at:
http://www.easyant.org/projects/easyant/files

Online documentation is accessible at : http://www.easyant.org/doc/

More information can be found on the Easyant website:
http://www.easyant.org/
Regards,

-- 
Jean Louis Boudart
Independent consultant
Project Lead http://www.easyant.org

Compress Antlib Release Plan

[Stefan Bodewig]

Hi,

the vote to release Apache Commoms Compress has finally started and even
if it should need one further re-roll (there has been a -1 and we'll see
whether this is enough to cancel the vote), the release should become
available in a week or two at most.

I intend to call for a vote on releasing the Compress Antlib with a
build based on the official Commons Compress release and the current
trunk of the Antlib as soon as the ACC release is published.  If you
want anything changed/added before the release of the Compress Antlib,
now is the time.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org