Re: [Tails-dev] automated tests

2012-06-19 Thread bertagaz
On Tue, Jun 19, 2012 at 01:53:38AM +0200, intrigeri wrote:
 
 This does look awesome!

Thanks I agree ;)

  It requires to use jruby, as sikuli is java. But there might be
  other ways to implement it.
 
 Just curious: is this requirement brought in by the sikuli gem?

Totally, because sikuli is java. I didn't choose this dep by taste ;) If
one day sikuli can be used with another language, I'm totally ready to
switch.


  You also need a Jruby = 1.6 environnement, which sadly isn't
  possible using debian packages yet
 
 Any hint / idea / pointer why?

See bug #636554, the maintainer seems to say back in september 2011 that
he is willing to upload jruby 1.6 *after* the testing migration. Not
sure what he means by that, probably after wheeze is released. Adding a
word on this bug is prolly a good idea, so if you want to you're welcome.

 
 Also, it's not clear to me what guest or host means in there:
   - on the VM guest or host who does the tests
   - everything happens on the host or guest where the build happen
 
 Could you please clarify this kind of sentences?

 I guess this means the system, that may be either a bare-metal one,
 or itself some kind of VM (with nested virtualization enabled), right?

Exactly, and that's why that was not easy to word correctly. I'll fix
this.

 Also, I'm unsure about where the build happen. Do you mean where
 the tests are run from, or anything else?

Yep, builds and tests are supposed to be started in the same system.

  It is also possible to add and remove different types of storages,
  and thus test persistence or filesystem modifications.
 
 Have you by chance found a way to *emulate* a USB 2.0 device in
 software? (qemu-kvm from current Debian testing/sid supports USB 2.0
 passthrough, but this is quite different as far as automated tests
 are concerned.)

Not really investigated in this area yet sorry. Apart from the
pass-through method I haven't seen anything else.

  but anyway you need this gems to be installed
 
 Do you want to {add references to,file} the corresponsding RFP bugs,
 or shall I?

As you wish. As long as jruby is  1.6 in Debian at least the sikuli gem
won't be packageable anyway. Might help to get a jruby upgrade.

  setup a basic VM
 
 Providing a guest XML skeleton would be perfect :)

Yeah, that's one of the questions brought by this tool. Might be usefull
if it contains a basic domain definition for sure. For consistency of
devices name for example.

To test different feature, there will be need for diffferent VMs
configurations, for example some with more memory for the memory wiping
on shutdown feature. There are mainly  two different ways to achieve this:

* programatically by using libvirt ruby bindings and use the functions to
attach/remove devices, modify the basic VM configuration, etc, which is
the current roughly implemented way.

* enable the tests to be shipped with a full domain definition, that would
be automatically loaded rather than a default one.

Between the two, tests could be shipped only with xml domain snippet
corresponding to the configuration changes to be done on the domain.

The second one would be the easiest, and might have the advantage to be a
totally stop and throw away this domain solution, so would help not to
modify too much the domain definition and keep consistency upon tests. But
there might be drawbacks too in this implementation.

 
  Set the DISPLAY env to something relevant
 
 What do you mean? Some unused X display?

Yes, on this unused display (so typically  :7), a Xvfb virtual X server
will be attached, and virt-viewer will be told to display the domain
display in full screen there. This is the trick to have sikuli detecting
the display (thgrough the $DISPLAY env variable) and using it for the
tests without having to play to much with vnc. :)

Thanks for the feedbacks.

bert.
___
tails-dev mailing list
tails-dev@boum.org
https://mailman.boum.org/listinfo/tails-dev


Re: [Tails-dev] automated tests

2012-06-19 Thread bertagaz
On Tue, Jun 19, 2012 at 01:59:14AM +0200, intrigeri wrote:

 Tiny remark: I think we could drop the last features component of
 features/cucumber/features/, so that
 features/cucumber/features/iceweasel/torified_browsing.feature
 becomes
 features/cucumber/iceweasel/torified_browsing.feature
 
 Makes sense?
 Do you want to do it, or shall I do it?

Yeah, I thought about that too after pushing. It requires to slightly
modify the logic that search for the iso in the vm_helper.rm though. I'll
do it.

bert.
___
tails-dev mailing list
tails-dev@boum.org
https://mailman.boum.org/listinfo/tails-dev


Re: [Tails-dev] automated tests

2012-06-19 Thread bertagaz
On Tue, Jun 19, 2012 at 01:39:17PM +0200, berta...@ptitcanardnoir.org wrote:
 On Tue, Jun 19, 2012 at 01:53:38AM +0200, intrigeri wrote:
  
  Have you by chance found a way to *emulate* a USB 2.0 device in
  software? (qemu-kvm from current Debian testing/sid supports USB 2.0
  passthrough, but this is quite different as far as automated tests
  are concerned.)
 
 Not really investigated in this area yet sorry. Apart from the
 pass-through method I haven't seen anything else.

Actually I just found http://www.linux-usb.org/gadget/ after a quick
search. I haven't yet really understood all of this project, but it seems
it has the feature to emulate USB at the kernel level without any
hardware. At least if you look at [1], it says:

/*
17  * This exposes a device side USB gadget API, driven by requests to a
18  * Linux-USB host controller driver.  USB traffic is simulated; there's
19  * no need for USB hardware.  Use this with two other drivers:
20  *
21  *  - Gadget driver, responding to requests (slave);
22  *  - Host-side device driver, as already familiar in Linux.
23  *
24  * Having this all in one kernel can help some stages of development,
25  * bypassing some hardware (and driver) issues.  UML could help too.
26  */

It requires USB_DUMMY_HCD to be configured in the kernel config, which
isn't in Debian kernel AFAIK.

Maybe I misunderstood the features, but might deserve some more digging
still.

bert.

[1] 
https://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=blob;f=drivers/usb/gadget/dummy_hcd.c;h=170cbe89d9f8ad47b9bb6ee54596521012c8de7f;hb=HEAD
___
tails-dev mailing list
tails-dev@boum.org
https://mailman.boum.org/listinfo/tails-dev


Re: [Tails-dev] [tor-dev] [GSoC] Tails Server

2012-06-19 Thread intrigeri
Hi,

jvoisin wrote (19 Jun 2012 01:53:43 GMT) :
 I am sorry but I won't be able to pursue/achieve my GSoC[1] for
 personal reasons that I prefer not disclose on a public
 mailing list.

Sorry about this.
I hope things will be better for you soon.
If we can help, please feel free to ask.

I'd rather not pressure you now, but if you want to come back and work
on Tails server with us at any later time, you are *much* welcome!

Take care,
--
  intrigeri
  | GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc
  | OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr.asc
___
tails-dev mailing list
tails-dev@boum.org
https://mailman.boum.org/listinfo/tails-dev


Re: [Tails-dev] automated tests

2012-06-19 Thread intrigeri
berta...@ptitcanardnoir.org wrote (19 Jun 2012 11:39:17 GMT) :
 Also, I'm unsure about where the build happen. Do you mean where
 the tests are run from, or anything else?

 Yep, builds and tests are supposed to be started in the same system.

Why?

(In the setup we had in mind for the upcoming autobuild/autotest
server, the builds and tests would be started from different systems,
due to incompatibility between the VirtualBox and KVM kernel modules.)
___
tails-dev mailing list
tails-dev@boum.org
https://mailman.boum.org/listinfo/tails-dev