Well, the problem is that there is a thread leak. It is not clear whether
the file creation succeeded.
It isn't that surprising that there is a thread leak since Hadoop code is
pretty cavalier about this and the Hadoop test cases don't check for this.
I think that there might be a way to signal that this is ok, possibly with
annotations. Somebody who knows the test randomizer framework would have
to comment on that.
On Mon, Jun 16, 2014 at 6:38 PM, Wei Zhang <w...@us.ibm.com> wrote:
> Thanks, Ted. I did try to do the fs.close(); The code looks like this:
> @Test
> *public* *void* simpleTest() *throws* IOException{
> Configuration conf = *new* Configuration();
> FileSystem fs = FileSystem.*get*(conf);
> Path filenamePath = *new* Path("/tmp/tmp.txt");
> FSDataOutputStream out = fs.create(filenamePath);
> out.close();
> fs.close();
> }
>
> It still generates the same error message.
>
> Thanks!
>
> Wei
>
> [image: Inactive hide details for Ted Dunning ---06/16/2014 09:29:28
> PM---Close the FileSystem object (fs). On Mon, Jun 16, 2014 at 6:2]Ted
> Dunning ---06/16/2014 09:29:28 PM---Close the FileSystem object (fs). On
> Mon, Jun 16, 2014 at 6:22 PM, Wei Zhang <w...@us.ibm.com> wrote
>
> From: Ted Dunning <ted.dunn...@gmail.com>
> To: "user@mahout.apache.org" <user@mahout.apache.org>,
> Date: 06/16/2014 09:29 PM
> Subject: Re: a seemingly benign test that fails MahoutTestCase
> ------------------------------
>
>
>
> Close the FileSystem object (fs).
>
>
>
>
> On Mon, Jun 16, 2014 at 6:22 PM, Wei Zhang <w...@us.ibm.com> wrote:
>
> > Hello Ted,
> >
> > Thank you very much for the reply! I have created a github gist that
> > records the error message at
> > https://gist.github.com/anonymous/21a2f36c995c1d39717b
> >
> > I am sorry that I forgot to attach the error log, it is something that
> > looks like this (just in case that github gist is displaying
> incorrectly):
> > Jun 16, 2014 6:07:10 PM
> > com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
> > WARNING: Will linger awaiting termination of 2 leaked thread(s).
> > Jun 16, 2014 6:07:30 PM
> > com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
> > SEVERE: 1 thread leaked from SUITE scope at
> > org.apache.mahout.wei.inc.util.HDFSUtilTest:
> > 1) Thread[id=15, name=org.apache.hadoop.hdfs.LeaseRenewer$1@7b968017,
> > state=TIMED_WAITING, group=TGRP-HDFSUtilTest]
> > at java.lang.Thread.sleep(*Native Method*)
> > at
> org.apache.hadoop.hdfs.LeaseRenewer.run(*LeaseRenewer.java:397*
> > )
> > at org.apache.hadoop.hdfs.LeaseRenewer.access$600(
> > *LeaseRenewer.java:69*)
> > at org.apache.hadoop.hdfs.LeaseRenewer$1.run(
> > *LeaseRenewer.java:273*)
> > at java.lang.Thread.run(Unknown Source)
> > Jun 16, 2014 6:07:30 PM
> > com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
> > INFO: Starting to interrupt leaked threads:
> > 1) Thread[id=15, name=org.apache.hadoop.hdfs.LeaseRenewer$1@7b968017,
> > state=TIMED_WAITING, group=TGRP-HDFSUtilTest]
> > Jun 16, 2014 6:07:30 PM
> > com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
> > INFO: All leaked threads terminated.
> >
> > Thanks!
> >
> > Wei
> >
> >
> > [image: Inactive hide details for Ted Dunning ---06/16/2014 07:50:30
> > PM---Attachments are stripped by the mailing list. Can you throw]Ted
> > Dunning ---06/16/2014 07:50:30 PM---Attachments are stripped by the
> mailing
> > list. Can you throw the stack trace onto a github gist or s
> >
> > From: Ted Dunning <ted.dunn...@gmail.com>
> > To: "user@mahout.apache.org" <user@mahout.apache.org>,
> > Date: 06/16/2014 07:50 PM
> > Subject: Re: a seemingly benign test that fails MahoutTestCase
> > ------------------------------
> >
> >
> >
> > Attachments are stripped by the mailing list. Can you throw the stack
> > trace onto a github gist or something?
> >
> >
> > On Mon, Jun 16, 2014 at 3:27 PM, Wei Zhang <w...@us.ibm.com> wrote:
> >
> > >
> > >
> > > Hello,
> > >
> > > I am wrting a simple Unit test, which extends MahoutTestCase. I am
> using
> > > Mahout 0.9 and Hadoop 1.0.3.
> > >
> > > The test is quite simple, it just creates an empty file on HDFS. The
> > entire
> > > coding is the following:
> > >
> > > @Test
> > > public void simpleTest() throws IOException{
> > > Configuration conf = new Configuration();
> > > FileSystem fs = FileSystem.get(conf);
> > > Path filenamePath = new Path("/tmp/tmp.txt");
> > > FSDataOutputStream out = fs.create(filenamePath);
> > > out.close();
> > > }
> > >
> > > However, this will fail the MahoutTestCase. In particular, the moment I
> > > added the line 4 (FSDataOutputStream out = fs.create(filenamePath);),
> > this
> > > test would fail.
> > > I am wondering can anyone give me some pointers regarding why this
> naive
> > > program would fail ?
> > > The error log is attached below.
> > >
> > > Thank you very much !
> > >
> > > Wei
> >
> >
>
>
