[dpdk-dev] dpdk pre-patch testing introduction
Hi Liu! I think this is a really, really great addition! On 2015-10-29 16:19, Liu, Yong wrote: > Currently the patch testing only run unit test and basic function test on > this platform. > It can make sure that new patch doesn't break original code and functions. > Due to coverage limitation, it can't verify the functionality of new patch. > If you want to verify your patch's new function, you need to check with > tester and perform specific testing on it. Speaking of coverage, I do have a few suggestions: 1. Build and test both with and without optimizations (build with EXTRA_CFLAGS="-O0"). While uncommon, I've seen code break in the non-optimized case because the compiler simply removes some unused code which would otherwise cause link errors. 2. I think it would be very good to collect code coverage for patches as well. (Yes, this is pushing my own tool) I've written a code coverage tool called kcov [1], which only needs debugging information (-g) in the binary to collect code coverage. It outputs results in various formats: lcov-style HTML, cobertura XML and can post directly to coveralls.io. It's also simple to run: kcov --include-pattern=dpdk /tmp/kcov-output ./testpmd [args...] To the mail report, I think it would be good to add information about code coverage increase/decrease with the patch and ideally which lines of the patch which were covered Coverage: 69.2% (+5 lines with the patch) Coverage for the patch: 5/7 lines both of these should be fairly simple to parse out from the XML output. I'd be happy to help you if think it's a good idea! // Simon [1] https://github.com/SimonKagstrom/kcov
[dpdk-dev] dpdk pre-patch testing introduction
2015-10-29 15:19, Liu, Yong: > So far, we only enabled one platform to run automatic testing. > This platform installed Fedora 20 with Linux 3.18.9 and Intel Niantic NIC. Thanks for offering this service. > Currently the patch testing only run unit test and basic function test on > this platform. > It can make sure that new patch doesn't break original code and functions. > Due to coverage limitation, it can't verify the functionality of new patch. > If you want to verify your patch's new function, you need to check with > tester and perform specific testing on it. > > Since platform limitation, we are not able to verify regression test on other > NIC, Linux kernel. > We plan to add more platforms and support different NIC and kernels in the > future. The mailing list for test report is open to others. Submitters are white-listed. Feel free to contribute with your own tests on your own platform. Some work is ongoing to track the test results from patchwork. https://lists.ozlabs.org/pipermail/patchwork/2015-July/001363.html https://lists.ozlabs.org/pipermail/patchwork/2015-July/001369.html
[dpdk-dev] dpdk pre-patch testing introduction
Dear all, Our validation team is working on pre-patch testing to support contiguous integration in DPDK community. I'd like to share the latest status and some details of our solution to you. The main target of pre-patch testing is supplement the health of each patch set as soon as possible. It will help both submitter and maintainer to know the status of patch set before it's merged. Some of you have received the email which was generated by this tool. Let me introduce it to you. Example email subject: |SUCCESS| pw 8162 ixgbe: remove useless fields in tx_offload on TCP checksum request The first item is the overall status of this patch set. The second item is the patchwork ID and its commit log. There are 4 status to report as the following: Malformed patch: Patch's format doesn't incorrect, not applied in latest branch SUCCESS : Pass all testing. ERROR : fail in some testing, error information can be found in report Not run : Failed to compile, not able to perform functional testing Email content included more detailed information of this patch set, like build failure log and failed cases log. If patch set modified files for document, the content will also include the status of "Doc Compilation". So far, we only enabled one platform to run automatic testing. This platform installed Fedora 20 with Linux 3.18.9 and Intel Niantic NIC. Currently the patch testing only run unit test and basic function test on this platform. It can make sure that new patch doesn't break original code and functions. Due to coverage limitation, it can't verify the functionality of new patch. If you want to verify your patch's new function, you need to check with tester and perform specific testing on it. Since platform limitation, we are not able to verify regression test on other NIC, Linux kernel. We plan to add more platforms and support different NIC and kernels in the future. Best regards, DPDK validation team
