Re: Lucene Jenkins slave out of disk
On Sun, Apr 24, 2011 at 12:21 PM, Uwe Schindler wrote: > I am merging realtime up to trunk now... thanks uwe! simon > > Uwe > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > >> -Original Message- >> From: Simon Willnauer [mailto:simon.willna...@googlemail.com] >> Sent: Sunday, April 24, 2011 12:06 PM >> To: dev@lucene.apache.org >> Subject: Re: Lucene Jenkins slave out of disk >> >> On Sun, Apr 24, 2011 at 9:53 AM, Uwe Schindler wrote: >> > Hi, >> > >> > JUHU! - Thanks. Now the test folder(s) (build/test, build/backwards/test) >> after running the builds only contain the test results and some empty dirs. >> > >> > Simon, if you merge that one we should be fine now! >> >> not until tuesday so feel free to merge! >> >> simon >> > >> > Uwe >> > >> > - >> > Uwe Schindler >> > H.-H.-Meier-Allee 63, D-28213 Bremen >> > http://www.thetaphi.de >> > eMail: u...@thetaphi.de >> > >> > >> >> -Original Message- >> >> From: Robert Muir [mailto:rcm...@gmail.com] >> >> Sent: Sunday, April 24, 2011 12:49 AM >> >> To: dev@lucene.apache.org >> >> Subject: Re: Lucene Jenkins slave out of disk >> >> >> >> On Sat, Apr 23, 2011 at 6:40 PM, Robert Muir wrote: >> >> > I thought i did this already, but maybe i screwed it up >> >> > >> >> >> >> Sorry, silly test bug... fixed in Revision: 1096249 >> >> >> >> - >> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For >> >> additional commands, e-mail: dev-h...@lucene.apache.org >> > >> > >> > >> > - >> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For >> > additional commands, e-mail: dev-h...@lucene.apache.org >> > >> > >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional >> commands, e-mail: dev-h...@lucene.apache.org > > > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Lucene Jenkins slave out of disk
I am merging realtime up to trunk now... Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Simon Willnauer [mailto:simon.willna...@googlemail.com] > Sent: Sunday, April 24, 2011 12:06 PM > To: dev@lucene.apache.org > Subject: Re: Lucene Jenkins slave out of disk > > On Sun, Apr 24, 2011 at 9:53 AM, Uwe Schindler wrote: > > Hi, > > > > JUHU! - Thanks. Now the test folder(s) (build/test, build/backwards/test) > after running the builds only contain the test results and some empty dirs. > > > > Simon, if you merge that one we should be fine now! > > not until tuesday so feel free to merge! > > simon > > > > Uwe > > > > - > > Uwe Schindler > > H.-H.-Meier-Allee 63, D-28213 Bremen > > http://www.thetaphi.de > > eMail: u...@thetaphi.de > > > > > >> -Original Message- > >> From: Robert Muir [mailto:rcm...@gmail.com] > >> Sent: Sunday, April 24, 2011 12:49 AM > >> To: dev@lucene.apache.org > >> Subject: Re: Lucene Jenkins slave out of disk > >> > >> On Sat, Apr 23, 2011 at 6:40 PM, Robert Muir wrote: > >> > I thought i did this already, but maybe i screwed it up > >> > > >> > >> Sorry, silly test bug... fixed in Revision: 1096249 > >> > >> - > >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For > >> additional commands, e-mail: dev-h...@lucene.apache.org > > > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For > > additional commands, e-mail: dev-h...@lucene.apache.org > > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional > commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Lucene Jenkins slave out of disk
On Sun, Apr 24, 2011 at 9:53 AM, Uwe Schindler wrote: > Hi, > > JUHU! - Thanks. Now the test folder(s) (build/test, build/backwards/test) > after running the builds only contain the test results and some empty dirs. > > Simon, if you merge that one we should be fine now! not until tuesday so feel free to merge! simon > > Uwe > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > >> -Original Message- >> From: Robert Muir [mailto:rcm...@gmail.com] >> Sent: Sunday, April 24, 2011 12:49 AM >> To: dev@lucene.apache.org >> Subject: Re: Lucene Jenkins slave out of disk >> >> On Sat, Apr 23, 2011 at 6:40 PM, Robert Muir wrote: >> > I thought i did this already, but maybe i screwed it up >> > >> >> Sorry, silly test bug... fixed in Revision: 1096249 >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional >> commands, e-mail: dev-h...@lucene.apache.org > > > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Lucene Jenkins slave out of disk
Hi, JUHU! - Thanks. Now the test folder(s) (build/test, build/backwards/test) after running the builds only contain the test results and some empty dirs. Simon, if you merge that one we should be fine now! Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Robert Muir [mailto:rcm...@gmail.com] > Sent: Sunday, April 24, 2011 12:49 AM > To: dev@lucene.apache.org > Subject: Re: Lucene Jenkins slave out of disk > > On Sat, Apr 23, 2011 at 6:40 PM, Robert Muir wrote: > > I thought i did this already, but maybe i screwed it up > > > > Sorry, silly test bug... fixed in Revision: 1096249 > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional > commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Lucene Jenkins slave out of disk
On Sat, Apr 23, 2011 at 6:40 PM, Robert Muir wrote: > I thought i did this already, but maybe i screwed it up > Sorry, silly test bug... fixed in Revision: 1096249 - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Lucene Jenkins slave out of disk
On Sat, Apr 23, 2011 at 6:27 PM, Uwe Schindler wrote: > Hi Robert, > > On Hudson there is still one very bad test, all others are cleaned up: > > [root@lucene > /home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/lucene/build/test/4]# > ls -lh > total 1286544 > -rw--- 1 hudson hudson 602M Apr 23 22:00 core.java.9439 > -rw--- 1 hudson hudson 656M Apr 23 22:00 core.java.9440 > -rw-r--r-- 1 hudson hudson 10K Apr 23 22:00 hs_err_pid9439.log > -rw-r--r-- 1 hudson hudson 11K Apr 23 22:00 hs_err_pid9440.log > -rw-r--r-- 1 hudson hudson 0B Apr 23 22:00 quiet.ant > > This one should be cleaned up or we should disable coredumps... How to do > this - for this test we need no coredumps, maybe we can add a parameter to > the ProcessBuilder? > we are not allowed to change the sysctl here... maybe i can change the test to use a CWD of its tempdir, so its corefiles will get deleted... I thought i did this already, but maybe i screwed it up - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Lucene Jenkins slave out of disk
On Sat, Apr 23, 2011 at 6:27 PM, Uwe Schindler wrote: > Hi Robert, > > On Hudson there is still one very bad test, all others are cleaned up: > > [root@lucene > /home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/lucene/build/test/4]# > ls -lh > total 1286544 > -rw--- 1 hudson hudson 602M Apr 23 22:00 core.java.9439 > -rw--- 1 hudson hudson 656M Apr 23 22:00 core.java.9440 > -rw-r--r-- 1 hudson hudson 10K Apr 23 22:00 hs_err_pid9439.log > -rw-r--r-- 1 hudson hudson 11K Apr 23 22:00 hs_err_pid9440.log > -rw-r--r-- 1 hudson hudson 0B Apr 23 22:00 quiet.ant > > This one should be cleaned up or we should disable coredumps... How to do > this - for this test we need no coredumps, maybe we can add a parameter to > the ProcessBuilder? i think this can be configured with sysctl, hopefully its per-jail and we can just set it... i'll take a look - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Lucene Jenkins slave out of disk
Hi Robert,
On Hudson there is still one very bad test, all others are cleaned up:
[root@lucene
/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/lucene/build/test/4]#
ls -lh
total 1286544
-rw--- 1 hudson hudson 602M Apr 23 22:00 core.java.9439
-rw--- 1 hudson hudson 656M Apr 23 22:00 core.java.9440
-rw-r--r-- 1 hudson hudson10K Apr 23 22:00 hs_err_pid9439.log
-rw-r--r-- 1 hudson hudson11K Apr 23 22:00 hs_err_pid9440.log
-rw-r--r-- 1 hudson hudson 0B Apr 23 22:00 quiet.ant
This one should be cleaned up or we should disable coredumps... How to do this
- for this test we need no coredumps, maybe we can add a parameter to the
ProcessBuilder?
That’s of course your JDK-crasher:
[...]
Stack: [0x7e4e9000,0x7e5e9000], sp=0x7e5e8160, free
space=3fc0001k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x71bc61]
j sun.misc.Unsafe.putAddress(JJ)V+0
v ~StubRoutines::call_stub
[...]
Uwe
-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de
> -Original Message-
> From: Robert Muir [mailto:rcm...@gmail.com]
> Sent: Saturday, April 23, 2011 7:56 PM
> To: dev@lucene.apache.org; simon.willna...@gmail.com
> Subject: Re: Lucene Jenkins slave out of disk
>
> On Sat, Apr 23, 2011 at 4:17 AM, Simon Willnauer
> wrote:
> >> Addition:
> >> I meant such code to be replaced:
> >>
> >> -indexDir = new File(workDir, "testIndex");
> >> +indexDir = _TestUtil.getTempDir("testIndex");
> >
>
> Thanks for merging Simon!
>
> also, for what its worth, we should at some point review these tests creating
> explicit directories.
>
> If a test wants to create an index, it can use newDirectory() or
> newFSDirectory(). The latter will only select filesystem-based
> implementations, never RAMDirectory, etc. When you use these methods, a
> unique temporary directory is automatically produced (and of course,
> deleted after the test if it passes).
>
> Most of these tests probably don't care what the directories actual name is...
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
> commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Lucene Jenkins slave out of disk
On Sat, Apr 23, 2011 at 4:17 AM, Simon Willnauer
wrote:
>> Addition:
>> I meant such code to be replaced:
>>
>> - indexDir = new File(workDir, "testIndex");
>> + indexDir = _TestUtil.getTempDir("testIndex");
>
Thanks for merging Simon!
also, for what its worth, we should at some point review these tests
creating explicit directories.
If a test wants to create an index, it can use newDirectory() or
newFSDirectory(). The latter will only select filesystem-based
implementations, never RAMDirectory, etc. When you use these methods,
a unique temporary directory is automatically produced (and of course,
deleted after the test if it passes).
Most of these tests probably don't care what the directories actual name is...
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Lucene Jenkins slave out of disk
On Sat, Apr 23, 2011 at 10:11 AM, Uwe Schindler wrote:
>> Hi,
>>
>>
>> > On Sat, Apr 23, 2011 at 9:47 AM, Uwe Schindler wrote:
>> > > Hi,
>> > >
>> > > Can you also check that all new tests in realtime use the new
>> > > _TestUtils API
>> > for getting an index dir? That would be nice.
>> >
>> > This only applies if we are getting an explicit index dir right?
>>
>> Yes!
>>
>
> Addition:
> I meant such code to be replaced:
>
> - indexDir = new File(workDir, "testIndex");
> + indexDir = _TestUtil.getTempDir("testIndex");
ok will do!
simon
>
> Uwe
>
>
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Lucene Jenkins slave out of disk
> Hi,
>
>
> > On Sat, Apr 23, 2011 at 9:47 AM, Uwe Schindler wrote:
> > > Hi,
> > >
> > > Can you also check that all new tests in realtime use the new
> > > _TestUtils API
> > for getting an index dir? That would be nice.
> >
> > This only applies if we are getting an explicit index dir right?
>
> Yes!
>
Addition:
I meant such code to be replaced:
-indexDir = new File(workDir, "testIndex");
+indexDir = _TestUtil.getTempDir("testIndex");
Uwe
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Lucene Jenkins slave out of disk
Hi, > On Sat, Apr 23, 2011 at 9:47 AM, Uwe Schindler wrote: > > Hi, > > > > Can you also check that all new tests in realtime use the new _TestUtils API > for getting an index dir? That would be nice. > > This only applies if we are getting an explicit index dir right? Yes! Uwe - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Lucene Jenkins slave out of disk
On Sat, Apr 23, 2011 at 9:42 AM, Uwe Schindler wrote: >> > any other branches (realtime? docvalues?) currently being tested by >> > hudson should merge up as soon as we can >> >> Thanks robert, I will merge RT now and commit... DocValues build is disabled >> currently I will make sure that I merge before reenabling it... > > Don't hurry, the FreeBSD machine hosting the Jail is down since about 18 hrs. > Major problems as it seems - or they are updating harddisks? *g* heh... tests already running.. so I will commit in a bit... maybe they give us a stronger machine :D simon > > Uwe > > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Lucene Jenkins slave out of disk
> > any other branches (realtime? docvalues?) currently being tested by > > hudson should merge up as soon as we can > > Thanks robert, I will merge RT now and commit... DocValues build is disabled > currently I will make sure that I merge before reenabling it... Don't hurry, the FreeBSD machine hosting the Jail is down since about 18 hrs. Major problems as it seems - or they are updating harddisks? *g* Uwe - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Lucene Jenkins slave out of disk
On Fri, Apr 22, 2011 at 5:13 PM, Robert Muir wrote: > On Fri, Apr 22, 2011 at 9:13 AM, Uwe Schindler wrote: >> Hi Robert, >> >> Thanks for pointing to that issue. Indeed the leftover test files in Lucene >> take approx. 3 GB per build. With our 9 builds that’s 30 GB - useless. If >> the tests clean up the thing successfully after running, we should be fine. >> > > I resolved this for trunk, branch_3x, and backwards. > > any other branches (realtime? docvalues?) currently being tested by > hudson should merge up as soon as we can Thanks robert, I will merge RT now and commit... DocValues build is disabled currently I will make sure that I merge before reenabling it... simon > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Lucene Jenkins slave out of disk
On Fri, Apr 22, 2011 at 9:13 AM, Uwe Schindler wrote: > Hi Robert, > > Thanks for pointing to that issue. Indeed the leftover test files in Lucene > take approx. 3 GB per build. With our 9 builds that’s 30 GB - useless. If the > tests clean up the thing successfully after running, we should be fine. > I resolved this for trunk, branch_3x, and backwards. any other branches (realtime? docvalues?) currently being tested by hudson should merge up as soon as we can - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Lucene Jenkins slave out of disk
Hi Robert, Thanks for pointing to that issue. Indeed the leftover test files in Lucene take approx. 3 GB per build. With our 9 builds that’s 30 GB - useless. If the tests clean up the thing successfully after running, we should be fine. If you are not able to solve this in near feature I would add the following command at the end of our shell script that runs the builds: find $WORKSPACE/checkout/lucene/build/test/ -type d -depth 1 | xargs rm -rf This would preserve test result files but delete all directories inside. Alternatively maybe the test task should do this after running tests? As noted before, running "ant clean" at the end of build process is not working as Jenkins need to collect test results. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Robert Muir [mailto:rcm...@gmail.com] > Sent: Friday, April 22, 2011 2:31 PM > To: dev@lucene.apache.org > Subject: Re: Lucene Jenkins slave out of disk > > On Thu, Apr 21, 2011 at 3:42 AM, Uwe Schindler wrote: > > OK, > > > > I was not aware that you took action, too. Because there is the > > message about Jenkins shutdown, I was assuming that you are fixing the > master. > > > > About the Lucene slave, the problem we have is: Our tests need lots of > > disk space for large indexes. > > and this is a known bug in our build that should really be fixed: > https://issues.apache.org/jira/browse/LUCENE-2796 > > i'll try to work on this today. > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional > commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Lucene Jenkins slave out of disk
On Thu, Apr 21, 2011 at 3:42 AM, Uwe Schindler wrote: > OK, > > I was not aware that you took action, too. Because there is the message > about Jenkins shutdown, I was assuming that you are fixing the master. > > About the Lucene slave, the problem we have is: Our tests need lots of disk > space for large indexes. and this is a known bug in our build that should really be fixed: https://issues.apache.org/jira/browse/LUCENE-2796 i'll try to work on this today. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Lucene Jenkins slave out of disk
On Thu, Apr 21, 2011 at 8:42 AM, Uwe Schindler wrote: > [root@lucene /home/hudson/hudson-slave/workspace]# df -h > Filesystem Size Used Avail Capacity > Mounted on > zroot/jails/lucene.zones.apache.org 69G 53G 17G 76% / > > I think the master is this time out of disk space. The disk space was cleared (or at least that's what Jenkins thinks). The slave is now running again. /niklas - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Lucene Jenkins slave out of disk
OK, I was not aware that you took action, too. Because there is the message about Jenkins shutdown, I was assuming that you are fixing the master. About the Lucene slave, the problem we have is: Our tests need lots of disk space for large indexes. Before the build, the workspace is complete cleared (at least for the nightly builds) and for the half-hourly builds it is svn-upped and then cleaned by ANT (ant clean). But at the end of the build the build script does not call "ant clean" again, because otherwise we would lost the test results. I am working on adding an additional task to our ant build scripts that will only clean up the test data that can be called after the build finished. This may clean up unneeded disk space after build so later running builds can reuse the space. I will talk to the other committers who know better where the temporary test files lie. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Niklas Gustavsson [mailto:nik...@protocol7.com] > Sent: Thursday, April 21, 2011 8:57 AM > To: bui...@apache.org > Cc: dev@lucene.apache.org > Subject: Re: Lucene Jenkins slave out of disk > > On Thu, Apr 21, 2011 at 8:42 AM, Uwe Schindler wrote: > > [root@lucene /home/hudson/hudson-slave/workspace]# df -h > Filesystem > > Size Used Avail Capacity Mounted on > > zroot/jails/lucene.zones.apache.org 69G 53G 17G 76% > > / > > > > I think the master is this time out of disk space. > > The disk space was cleared (or at least that's what Jenkins thinks). > The slave is now running again. > > /niklas - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Lucene Jenkins slave out of disk
Hi Niklas, [root@lucene /home/hudson/hudson-slave/workspace]# df -h Filesystem SizeUsed Avail Capacity Mounted on zroot/jails/lucene.zones.apache.org 69G 53G 17G76%/ I think the master is this time out of disk space. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Niklas Gustavsson [mailto:nik...@protocol7.com] > Sent: Thursday, April 21, 2011 8:21 AM > To: dev@lucene.apache.org > Cc: bui...@apache.org > Subject: Lucene Jenkins slave out of disk > > Hi > > The box running the Jenkins slave for Lucene is out of disk space again. > Please check. > > /niklas - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
