Hi Cai & Daniel,
On Thu, 2008-10-23 at 12:01 -0700, CAI Qian wrote:
> Hi,
>
> --- Daniel Gollub <[EMAIL PROTECTED]> wrote:
>
> > On Thursday 23 October 2008 19:47:56 CAI Qian wrote:
> > > Hi,
> > >
> > > Ptrace06 has lots of issues. It fails to build on IA-64, unable to
> > > compile on earlier Kernels
> > > (more details on those two issues could be found on previous threads) and
> > > there is new issue I
> > > have just spotted.
> > >
> > > In spawn_ptrace_child.h,
> > >
> > [...]
> >
> > >
> > > The parent returns from the function in which vfork() was called. As the
> > > results, the test
> > result
> > > is random on x86-64,
> > >
> > > ptrace06 1 PASS : ptrace(PEEKDATA, ..., (nil), (nil)) failed as
> > > expected
> > >
> > > or
> > >
> > > ptrace06 1 FAIL: ptrace(PEEKDATA, ..., (nil), (nil)) expected errno
> > > EIO or EFAULT;
> > actual: No
> > > such process
> > >
> > > or
> > >
> > > ptrace06 1 FAIL: ptrace(PEEKDATA, ..., (nil), (nil)) returned 0
> > > instead of -1
> > >
> > > According vfork() manpage, the code is does not look right.
> > >
> > > "The vfork() function has the same effect as fork(2), except that the
> > > behavior is undefined if
> >
> > > the process created by vfork() either modifies any data other than
> > > a variable of type
> > pid_t
> > > used to store the return value from vfork(), or returns from the function
> > > in which vfork() was
> > > called,"
> > >
> > > Also, it leaves lots of processes after the test completes.
> > >
> > > qcai 4503 0.0 0.0 4092 408 pts/8 S 01:22 0:00
> > > ./ptrace06 child
> > > qcai 4541 0.0 0.0 4092 408 pts/8 S 01:23 0:00
> > > ./ptrace06 child
> > > qcai 4584 0.0 0.0 4092 408 pts/8 S 01:23 0:00
> > > ./ptrace06 child
> > > qcai 4606 0.0 0.0 4092 404 pts/8 S 01:23 0:00
> > > ./ptrace06 child
> >
> > I have seen something similar in the past with times03 (iirc) which was
> > about
> > about a compiler optimization which could tight assembler infinite loop and
> > interfered with further running tests -> bad impact on testresults.
> > (This got already fixed in beginning of the year...)
> >
> > Is there already some code in runltp to catch such processes and abort the
> > entire testrun? Test processes leaking out of the scope of the testdriver
> > is a pretty serious thing - isn't it? We should definitely ban such broken
> > tests and prepare runltp to detect such processes? In my opinion aborting
> > the testrun immediately is better to have wrong/invalid testresults...
> >
>
> Agree. We need to make sure we tidy up things from the previous test before
> to run the next one.
>
> >
> > > The test has been disabled in Makefile at the moment, but I would like to
> > > discuss how we could
> > > handle the similar case in the future. In this case, it is unnecessary to
> > > remove it from CVS
> > > entirely, but it is kind of untidy and confusion to users to leave in the
> > > test directory, but
> > > never been compiled.
> >
> > Fully agree.
> >
> > I'm pretty new to LTP development, but what about the open-posix-testsuite?
> > It's also disable by default - looking into open-posix-testsuite show also
> > definitely some issues ...
> >
> > (And those "grep -v" are really not very obvious - i have to admin i didn't
> > actively checked the commit logs or documentation - maybe i'm missing some
> > obvious documentation about this...)
> >
>
> I don't know either.
>
> > > How about to create a separate directory (testcases/working_in_progress?)
> > > to
> > > move all those broken test code there, and each of them has an associate
> > > file to document why
> > it
> > > is here and what need to be done to move it to the test directory it
> > > should belong?
> >
> > Good idea!
Yes, this can be done. We can have something like
testcases/broken_cases/*. Move the very much troubling guys like the
present ptrac06 to here. And once it is fixed, it can be moved back.
Having said that, we have to do the following things always:
1) First, we need to decide whether somebody is really troubling. In
other words, how do we decide the degree of trouble. A test may have a
little problem on only an exclusive arch, but may be perfectly OK in
others. In such a case, it is not a good candidate to move. However, i
am hopeful that we can agree to decide on this,
2) Both the to-and-fro movement needs to be addressed through
appropriate patches: like disabling build/install in Makefile and
diabling run in runtest/* file(s). Similarly enabling patches in these
places when they are restored,
> >
> > How to handle testcases which build fine on all architectures and all
> > kernel versions - but can't trigger a certain error-path for a kernel
> > interface for a specific kernel version? Or not on specific Architectures?
> > Or combined: build on all kernels, but is not testable on 2.6.27 on x86_64?
> >
> > Example: testcases/open_posix_testsuite/conformance/interfaces/mmap/31-1.c
> > I'm looking into this one right now - i guess it's not possible to trigger
> > EOVERFLOW on x86_64 with current (or even any?) kernel on x86_64.
> >
>
> That is something missing in LTP. In my opinion, each test or set of tests
> need to indicate which
> architectures and Kernel versions it supports, and make sure only compile and
> run on those
> supported systems. So, as long as it is written somewhere, people could just
> contribute a test to
> support only one architecture or Kernel version that she/he cares about
> without worrying about
> having to provide support on all architectures and Kernel versions that LTP
> aims to.
We already have this trend in LTP. The READMEs say on which archs they
can run. The Makefile(s) are written to trigger build in such archs
only. And when you run them, they say they are not supported arch. That
should be a good referral. Isn´t it ?
Having said that, i fully agree that such information, like; on which
archs/kernel versions, headers/glibc version, headers they should run;
is missing from most of the test cases except the most recent ones. So,
we can have a strategy from now onwards:
1) Whoever submits a patch on certain set of test case(s), updates that
Information as well, as, we can suppose that he has worked on that test
case, on his interested archs, and, has the relevant information. People
working/interested on other archs can update when they work on the same
case in future,
2) Any new test case(s) coming in should have this mandatory information
in them. So, what all information you would like to have been documented
there?
- Architectures supported,
- Kernel Versions supported/disbaled thereafter,
- glibc version,
- glibc headers,
- kernel headers,
- fs type,
- etc,
Add to the above list, and i will make it mandatory for others to
provide such information whenever they submit a patch.
Regards--
Subrata
>
> Cai Qian
>
> > best regards,
> > Daniel
> >
>
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Ltp-list mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/ltp-list
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list
