Re: test buffering causes OOM errors

2012-07-23 Thread Dawid Weiss
Just for completeness -- something I forgot about. There is one more option that I forgot about -- that ANT target is meant to simplify splitting suites into multiple JVMs and providing nicer integration with the randomized runner (seed passing etc). You can still use a regular ANT's to run the

Re: test buffering causes OOM errors

2012-07-21 Thread Dawid Weiss
> +1, I also miss simple streamed output option when running a test. > > And it's fine if it only works if you run one test, one JVM, etc. There is an issue for 1-jvm streaming on github. Like I said -- it is possible to decrease memory consumption (stdout buffering) although there will always be

Re: test buffering causes OOM errors

2012-07-21 Thread Dawid Weiss
> +1, I also miss simple streamed output option when running a test. > > And it's fine if it only works if you run one test, one JVM, etc. There is an issue for 1-jvm streaming on github. Like I said -- it is possible to decrease memory consumption (stdout buffering) although there will always be

Re: test buffering causes OOM errors

2012-07-21 Thread Michael McCandless
On Sat, Jul 21, 2012 at 5:49 AM, Yonik Seeley wrote: > Logs are most useful in big stress tests where things are not easily > reproducible or debuggable with a debugger - and these also tend to > produce a lot of logging. +1, I also miss simple streamed output option when running a test. And it

Re: test buffering causes OOM errors

2012-07-21 Thread Dawid Weiss
I am on mobile but I will try to justify the design decisions I made later on. As for now, quickly, tests.iters has been fixed and should not oom unless you run with many thousands of repetitions (if so it is better to loop manually). As for the oom on large sysouts - incease ant memory or redire

Re: test buffering causes OOM errors

2012-07-21 Thread Yonik Seeley
On Sat, Jul 21, 2012 at 5:36 AM, Dawid Weiss wrote: > Full stack trace would me more helpful Full stack trace below. > but I think the problem is that solr > tests emit megabytes of data to the console and this has to be buffered and > passed to report listeners. I've not looked into the test f

Re: test buffering causes OOM errors

2012-07-21 Thread Dawid Weiss
Full stack trace would me more helpful but I think the problem is that solr tests emit megabytes of data to the console and this has to be buffered and passed to report listeners. It is possible to spill to disk of course but not a trivial refactoring. An easy workaround for now would be to redir

test buffering causes OOM errors

2012-07-21 Thread Yonik Seeley
Is there a way we can write to a file instead of buffering test output in memory? This has been causing exceptions for me lately. [junit4:junit4] Event bus dispatch error: java.lang.OutOfMemoryError: Java heap space [junit4:junit4] java.lang.OutOfMemoryError: Java heap space [junit4:junit4]