On 2012.10.01 16:37, Sean McBride wrote: > But that's not a reason to have *no* unit testing. :)
Yeah, and the wiki Test page should be proof enough that I don't want that either. All I'm trying to point out is that it's unrealistic to expect us to go through the whole gamut of what we should ideally test, and therefore keep in mind that we are likely to keep receiving complaints from people who spot problems in a path we didn't have the luxury to test, and who may expect libusbx to either have enough volunteers to provide some semi-comprehensive testing with each release, or the for-profit backing up that's required otherwise. > Again, I propose ctest & cdash. It's easy for community members to submit > compilation & test results of all sorts of configurations, as seen here for > example: > > <http://open.cdash.org/index.php?project=CMake> Anything that can automate our testing tasks is fine with me, but with the following possible caveats/questions: - CDash seems to be quite tied up to CMake, whereas we're using autotools, and we've seen after Vitali's departure that the demand to justify CMake maintenance wasn't really there. I'd rather avoid introducing a CMake dependency and a new maintenance/support requirement in libusbx unless there is a strong demand for it. Also, from what I gather, it seems to duplicate some of the features we will get from Jenkins (when we finally find some spare time to properly set the thing up). - It seems like the bread and butter of CDash/CTest is checking builds (which is something we want to do of course) as well as pure software unit testing, whereas the area where we probably want to concentrate our testing is tied to specific hardware access, and would require intervention (driver switching, plug a device) that can't exactly be automated. The reasoning is that issues that are not tied to hardware access, such as compilation breaking on a specific platform or even breakage of apps such as usbutils and OpenOCD, are going to be reported fairly extensively compared to ones tied to a specific hw conf or type of hw access => with our current lack of resources, it may be more more helpful to our users to concentrate on the latter. And if that's the case, I'm not sure if there's much to be gained from just asking someone to manually place a checkbox in a static table during RCs. > We would contribute nightly builds on 10.5, 10.6, 10.7, 10.8, and Ubuntu 12, > using different versions of gcc and clang. Notwithstanding the (possible) reservations expressed above, if there's a way for you to provide the above, without introducing a dependency on CMake, I'm all for it! Now, if a dependency on CMake is preferable, I'm not closing the door on either (After all, I did add Vitali's CMake support in my branch when he proposed it), but I'd like to hear what others have to say. Regards, /Pete ------------------------------------------------------------------------------ Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html _______________________________________________ libusbx-devel mailing list libusbx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libusbx-devel
