On Sun, 20 Feb 2011 19:51:38 -0600 Ravenlock <ravenl...@ravenlock.us> said:
> On 02/19/2011 08:10, Carsten Haitzler (The Rasterman) wrote: > > On Mon, 14 Feb 2011 12:17:03 +0100 Lionel Orry <lionel.o...@gmail.com> said: > > > > just fyi - CI is one of the lesser worries we have. so let's not make this > > more that it is - builds for us are stable and well tested. i rebuild efl > > between 1-4 times a day. sometimes much more. between the developers we > > have little issue with rebuilds. > > > > now let's get to the core of this - REBUILDS arent a problem. i rebuild efl > > and then some in about 6 mins on my desktop/laptop. i dont use distcc - > > nothing beyond the single cpu there. some smart Makefile to allow parallel > > builds between libs. so this isn't an issue. making sure things build is > > the least of our issues. > > > > The proper way (imho), to fashion a CI builderver would be to guarantee > it does a clean, an update of sources, and a full build each time. This > helps catch a suite of build issues not necessarily seen on developer > machines. > > > what we need to track is BUGS. when someone introduces a bug - the longer > > it is not found, the harder it is to fix later. this means we arent really > > about rebuilds. we are about TESTING every change that we can. we have SOME > > test suites right now - expedite is an automated one for evas, we have some > > for eina as well ans a bit for ecore - but they are mostly very thing and > > done test a lot. elementary has a test, but its interactive. > > Unit tests are what help. There are plenty of unit test automation > tools out there. The unit tests should be performed on the build server > (out of reach of developer hands) each time there is a build. They are > quick simple tests which garuntee things like the API is not broken, > among other things. Now that EFL has been released and the API written > in stone, Unit tests can make sure the API remains rock solid. > > Using TDD one would write their tests first and then write code until > the tests succeed. Its an interesting exercise. Never too late to > start. :P > > Maybe these are the test suites you mention. I've not looked at > expedite. But in either case, unit test can be very helpful when done > properly. the "test harness" is by far and wide the least problem. the BIg problem is writing all the TESTS. everyone focuses on "but use check" as if it's a silver bullet that suddenly means your code is all tested. these are simple small harnesses to go RUN a bunch of tests - 99% of the work is doing the TESTS, not in the harness. thats my point. > > what we need to do is work on fleshing out tests where they are mostly good > > (expedite for example) so they test more or everything - and can automate > > the test. then for others create tests ANd find ways to automate them - > > increasingly we will need to find a way to automate gui interaction as > > thats a huge amount of what we do - and then verifying that the results of > > the interaction (logical and display) are "right". > > Unit testing GUI is very challenging. Doable, but not always easy. immensely time consuming is the word. possible -but hard and takes a lot of work and even need fuzzy tests to allow for varying results (to a degree). > > > > this here has nothing to do with build bots, hosts, jenkins etc. but > > requires building and improving other infra and tools. we should be doing > > that long before we care about the infra to run those tests. > > I think it has a lot (maybe everything) to do with a build > server/system. I don't know about jenkins or what not. Actually after > hearing that jenking or the other mentioned actually ran on developer > slave machines, I was a bit turned off. I personally prefer something > that runs out of reahc of the developer. Something to remain pristine > and free of molestation. It helps remove the "Works For Me" (tm) out of > the equation. then it's "it works for the server it runs on" and not someone else. all you do is create "yet another developer machine" to run it on. this is helpful if developer machines are in short supply and dont have the time, power, resources to run tests. that is not the case for us. what we lack are all the tests. :) we have some, here and there. nowhere near as many as we should have. > > > > so... who is volunteering to work on the tests? :) > > > >> On Sat, Feb 12, 2011 at 2:32 AM, Ravenlock <ravenl...@ravenlock.us> wrote: > >>> Can FreeBSD users play too, with the majority of ya'll running linux? > >>> > >> In the case of Buildbot and Jenkins, the concept of master/slave is > >> central and should definitely be considered seriously. That means not > >> only a build server, but distributed build over a build farm of > >> computers with different OSes. > >> > >> I know Jenkins better so I'll talk a little bit about that. > >> > >> Say the master is a Linux box. Fine, it runs the java container and > >> Jenkins application as the server machine, and it can also be used as > >> a build machine by following a task to try to build a library or > >> application, or generate doxygen, or get code coverage / unit testing > >> / whatever a script or a plugin can do for us. > >> > >> But this master can also trigger a build on a distant machine, should > >> it be running any Unix (in that case, ssh connection between the 2 > >> machines is the easiest and painless way, with a dedicated > >> 'hudson_slave' user for example for security) or even Windows > >> (commands issued via JLNP or a windows service). > >> > >> One of the machines could be running a 'BSD OS. > >> > >> Going further: we only want one machine because the other ones are > >> owned by the devs, they bite and don't want a distant robot to execute > >> commands and take up their CPU. > >> > >> Fine again. Let's virtualize. Jenkins is able to connect, via its > >> plugins, to various virtual machine interfaces (vmware, virtualbox). > >> QEmu is easy to automate so no plugin needed. So we can simply run > >> different virtual machines on the server, on demand, when a build is > >> triggered on them. It is transparent from the Jenkins interface, the > >> virtual machines appear like other physical nodes. > >> > >> So, many possibilities. FreeBSD users will have their playground too, > >> don't worry. :) > >> > >> Lionel > >> > >> ------------------------------------------------------------------------------ > >> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > >> Pinpoint memory and threading errors before they happen. > >> Find and fix more than 250 security defects in the development cycle. > >> Locate bottlenecks in serial and parallel code that limit performance. > >> http://p.sf.net/sfu/intel-dev2devfeb > >> _______________________________________________ > >> enlightenment-devel mailing list > >> enlightenment-devel@lists.sourceforge.net > >> https://lists.sourceforge.net/lists/listinfo/enlightenment-devel > >> > > > > > > > -- > Regards, > Ravenlock > > -- ------------- Codito, ergo sum - "I code, therefore I am" -------------- The Rasterman (Carsten Haitzler) ras...@rasterman.com ------------------------------------------------------------------------------ The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb _______________________________________________ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
