On Tue, Feb 03, 2009 at 11:29:32AM -0600, Adam Litke wrote: > Adding Mel Gorman and Eric Munson to this discussion since they are the > team working on libhugetlbfs now.
Content-Description: Forwarded message - Re: [Fwd: [RFC] [HUGETLB tests] Should we scrap them for libhugetlbfs ??] > Date: Tue, 3 Feb 2009 16:58:18 +0000 > From: Mel Gorman <[email protected]> > To: Adam Litke <[email protected]> > Subject: Re: [Fwd: [RFC] [HUGETLB tests] Should we scrap them for > libhugetlbfs ??] > > Howdy, you didn't actually cc me to them so I didn't respond back to them > directly. However, feel free to forward this mail to them cc'd on my my, > Eric's and Larry's IBM internal addresses. > > On Tue, Feb 03, 2009 at 08:20:45AM -0600, Adam Litke wrote: > > > > > Hi, > > > > I had long been hearing about the LTP hugetlb test cases being broken > > and no more able to conform to recent changes to hugetlbfs. > > AFAIK, the libhugetlbfs developers have not been using the LTP-based > tests. For distribution testing, we always use the regression tests that > are part of the library on the assumption their coverage of code and > corner cases was higher. > > > So, every > > time an issue crops up, we need to do some surgery to restore those test > > cases. And the present test cases need a major overhaul from some expert > > in hugetlbfs, who can find time to fix them. > > > > Well, essentially you would be reproducing the expertise that is > codified in the libhugetlbfs regression tests. They have been built up > over time where a kernel bug was found, a unit test added, fixed and the > unit test left in place. > > > However, we have a good testsuite on hugetlbfs from: > > http://sourceforge.net/projects/libhugetlbfs, > > which gets updated continuously and improved upon. I did a comparative > > code coverage (on 2.6.28) for: > > 1) Hugetlb tests in LTP, and, > > (http://ltp.cvs.sourceforge.net/viewvc/ltp/ltp/testcases/kernel/mem/hugetlb/), > > 2) Hugetlb tests in LIBHUGETLBFS > > (http://sourceforge.net/projects/libhugetlbfs), > > > > LIBHUGETLBFS tests has a better code coverage (15.9%) over LTP-Hugetlb > > (12.4%). > > This is very interesting. 15.9% of what? The hugetlbfs code only or the > kernel overall? Indeed. It would also be interesting to get the combined coverage of both test sets to see if the libhugetlbfs tests cover strictly more area than the LTP ones, or if there's something that the LTP tests cover which the library ones don't (in which case we should extend the libhugetlbfs suite to include it). > > If it does make sense, then i can go ahead and see how i can > > remove the present tests and integrate the new ones, and how to keep > > them updating from the latest libhugetlb releases. > > I would suggest you check exactly what LTP is testing and make sure the > the equivalent is being covered by the libhugetlbfs tests in case we are > missing something. > > There are three large problems with a straight port of the tests to LTP. > > 1. The tests were not written with LTP in mind so there will be API > issues around things like reporting a test failure. > > 2. Some of the tests are known to fail if the kernel is old or the > pagesize being used is 16GB. It's a planned workitem to identify where > expected failures occur and report EXPECTED_FAIL instead of FAIL > but it's not done at the moment. > > 3. Some tests depend on libhugetlbfs functionality. Some of the tests call > APIs > that are available in the library to make sure the library is functioning > as expected. Some are the public API like calling gethugepagesize() > (manual page in library) but others are parts of an internal API that > the tests link to directly to make sure the library internals are solid. > These will be less straight-forward to port. You could just restrict > yourself to the tests that test kernel interfaces though and that > would be useful. > > One potential way around this is for the LTP tests to run the libhugetlbfs > tests instead of writing their own. That way the one set of tests are being > used. It would report FAIL if any one of the libhugetlbfs tests reported > failure. This would provided added incentive for the libhugetlbfs people to > people to get their "expected failure" detection right. > > > Can you kindly reply with a Signoff to this mail, if you do not have > > objection in including the tests to LTP from libhugetlbfs ? > > I can't speak for Adam obviously but I can't see any reason to object to the > tests being ported to LTP but check out any licensing issues. libhugetlbfs > is LGPL and oddly that would apply to the test cases as well. If LTP is GPL > or something else, that might be a problem - don't ask me, I'm not a lawyer. LGPL code into a GPL project is fine (by design, the LGPL is strictly less restrictive than the GPL). > If you are looking for a libhugetlbfs developer to actually port the tests > to LTP, that's a different issue. There are only two active libhugetlbfs > developers within IBM and porting tests to LTP is not on their list of > planned items at the moment. It would be difficult to justify why a port > to LTP would take a higher priority over other proposed work items and what > the benefits would be but the final call is not mine of course. > > Some sort of wrapper that downloaded a stable version of libhugetlbfs > and then whatever the latest version is and testing both of those is an > interesting possibility though. It could even be done in addition to the > existing LTP tests instead of a replacement. -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson ------------------------------------------------------------------------------ Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM) software. With Adobe AIR, Ajax developers can use existing skills and code to build responsive, highly engaging applications that combine the power of local resources and data with the reach of the web. Download the Adobe AIR SDK and Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com _______________________________________________ Ltp-list mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ltp-list
