Re: [Tails-dev] Automated tests specification

2016-01-06 Thread intrigeri
intrigeri wrote (05 Jan 2016 23:44:50 GMT) : > ⇒ bertagaz, please find (or create) that ticket, and make sure it > references the starting point of this thread for each month in the > list archives: Done: https://labs.riseup.net/code/issues/9719 ___

Re: [Tails-dev] Automated tests specification

2016-01-05 Thread intrigeri
Hi, it seems like the sub-thread I'm replying to was a good discussion about the Jenkins / Redmine interaction. It contains plenty of good thinking. It didn't end up with actionable conclusions, which is not a surprise, nor a failure I think: it was deliberately about stuff we did not expect

Re: [Tails-dev] Automated tests specification

2015-09-22 Thread intrigeri
intrigeri wrote (15 Sep 2015 08:58:48 GMT) : > bertagaz wrote (03 Sep 2015 08:44:01 GMT) : >> On Wed, Sep 02, 2015 at 01:12:39PM +0200, intrigeri wrote: >>> I think that's not a blocker for the first iteration, though: if videos >>> are added between Oct. 15 and Jan. 15, I'm happy :) >>> >>>

Re: [Tails-dev] Automated tests specification

2015-09-15 Thread intrigeri
bertagaz wrote (03 Sep 2015 08:44:01 GMT) : > On Wed, Sep 02, 2015 at 01:12:39PM +0200, intrigeri wrote: >> I think that's not a blocker for the first iteration, though: if videos >> are added between Oct. 15 and Jan. 15, I'm happy :) >> >> bertagaz, time to create tickets that sum this up, or do

Re: [Tails-dev] Automated tests specification

2015-09-03 Thread bertagaz
Hi, On Wed, Sep 02, 2015 at 01:12:39PM +0200, intrigeri wrote: > > I think that's not a blocker for the first iteration, though: if videos > are added between Oct. 15 and Jan. 15, I'm happy :) > > bertagaz, time to create tickets that sum this up, or do we need > more discussion? Done, in

Re: [Tails-dev] Automated tests specification

2015-09-02 Thread anonym
On 09/01/2015 06:57 PM, anonym wrote: > On 09/01/2015 12:04 PM, intrigeri wrote: >> bertagaz wrote (28 Aug 2015 14:24:51 GMT) : >>> and might take quite a bit of disk space to store. >> >> ... and smaller (#10001). I'm curious how much space a full set of >> test suite videos take. > > I'll try

Re: [Tails-dev] Automated tests specification

2015-09-02 Thread anonym
On 09/02/2015 11:11 AM, anonym wrote: > Failing Scenarios: > cucumber features/time_syncing_bridges.feature:29 # Scenario: Clock way > in the past in bridge mode > cucumber features/time_syncing_bridges.feature:45 # Scenario: Clock way > in the future in bridge mode > cucumber

Re: [Tails-dev] Automated tests specification

2015-09-02 Thread intrigeri
hi, anonym wrote (02 Sep 2015 09:11:43 GMT) : > I got a video that was 309 MiB large [...] Assuming (very wild guess) that the size gain brought by the proposed changes from #10001 can be extrapolated, this could become 117 MiB. If that's mostly correct, then with an aggressive enough cleaning

Re: [Tails-dev] Automated tests specification

2015-09-02 Thread intrigeri
anonym wrote (01 Sep 2015 16:57:05 GMT) : > Yes, per-scenario videos would be great (my plan was to do this when we > have #8947, but whatever, nothing prevents us from having it now). They > would be more useful than per-feature videos, and actually easier to > implement AFAICT. Please file a

Re: [Tails-dev] Automated tests specification

2015-09-02 Thread anonym
On 09/02/2015 01:12 PM, intrigeri wrote: > hi, > > anonym wrote (02 Sep 2015 09:11:43 GMT) : >> I got a video that was 309 MiB large [...] > > Assuming (very wild guess) that the size gain brought by the proposed > changes from #10001 can be extrapolated, this could become 117 MiB. > If that's

Re: [Tails-dev] Automated tests specification

2015-09-02 Thread intrigeri
hi, bertagaz wrote (02 Sep 2015 10:41:59 GMT) : > On Tue, Sep 01, 2015 at 06:59:09PM +0200, anonym wrote: >> On 09/01/2015 12:23 PM, intrigeri wrote: >> > bertagaz wrote (26 Aug 2015 17:52:26 GMT) : >> Since pushing stuff into the branch after this field has been set to >> true invalidates the

Re: [Tails-dev] Automated tests specification

2015-09-02 Thread bertagaz
On Tue, Sep 01, 2015 at 06:59:09PM +0200, anonym wrote: > On 09/01/2015 12:23 PM, intrigeri wrote: > > bertagaz wrote (26 Aug 2015 17:52:26 GMT) : > >> On Wed, Aug 26, 2015 at 03:38:19PM +0200, anonym wrote: > >>> The current proposal seems to be to only start the automated test run of > >>> a

Re: [Tails-dev] Automated tests specification

2015-09-02 Thread bertagaz
Hi, On Tue, Sep 01, 2015 at 12:04:39PM +0200, intrigeri wrote: > bertagaz wrote (28 Aug 2015 14:24:51 GMT) : > > > > But then, often the screen capture are enought to identify why > > a step failed to run. > > In my experience, sometimes, what would help understanding the problem > has

Re: [Tails-dev] Automated tests specification

2015-09-02 Thread bertagaz
Hi, On Tue, Sep 01, 2015 at 06:57:05PM +0200, anonym wrote: > On 09/01/2015 12:04 PM, intrigeri wrote: > > bertagaz wrote (28 Aug 2015 14:24:51 GMT) : > >> I've also added a new section about the result to keep: > > > >> ## What kind of result shall be kept > > > >> The test suite produces

Re: [Tails-dev] Automated tests specification

2015-09-01 Thread anonym
On 09/01/2015 12:23 PM, intrigeri wrote: > bertagaz wrote (26 Aug 2015 17:52:26 GMT) : >> On Wed, Aug 26, 2015 at 03:38:19PM +0200, anonym wrote: >>> The current proposal seems to be to only start the automated test run of >>> a feature branch when it is marked "Ready for QA". This has overloaded

Re: [Tails-dev] Automated tests specification

2015-09-01 Thread anonym
On 09/01/2015 12:04 PM, intrigeri wrote: > Hi, > > bertagaz wrote (28 Aug 2015 14:24:51 GMT) : >> On Wed, Aug 26, 2015 at 07:46:17PM +0200, anonym wrote: >> I've also added a new section about the result to keep: > >> ## What kind of result shall be kept > >> The test suite produces different

Re: [Tails-dev] Automated tests specification

2015-09-01 Thread intrigeri
Hi, bertagaz wrote (28 Aug 2015 14:24:51 GMT) : > On Wed, Aug 26, 2015 at 07:46:17PM +0200, anonym wrote: > I've also added a new section about the result to keep: > ## What kind of result shall be kept > The test suite produces different kind of artifacts: logfiles, screen > captures for

Re: [Tails-dev] Automated tests specification

2015-09-01 Thread intrigeri
bertagaz wrote (26 Aug 2015 17:52:26 GMT) : > On Wed, Aug 26, 2015 at 03:38:19PM +0200, anonym wrote: >> The current proposal seems to be to only start the automated test run of >> a feature branch when it is marked "Ready for QA". This has overloaded >> the meaning of this status so it no longer

Re: [Tails-dev] Automated tests specification

2015-08-28 Thread bertagaz
Hi, On Wed, Aug 26, 2015 at 07:46:17PM +0200, anonym wrote: On 08/26/2015 07:21 PM, bertagaz wrote: The rational behind my proposal was that it would at least raise the issue if there were some external changes that breaks the build of this feature branch (mostly, changes in APT/Debian).

Re: [Tails-dev] Automated tests specification

2015-08-26 Thread bertagaz
Hi, Many thanks for your deep review and opinion share. More below. On Wed, Aug 26, 2015 at 02:00:25PM +0200, anonym wrote: On 07/01/2015 07:19 PM, intrigeri wrote: bertagaz wrote (25 Jun 2015 09:41:23 GMT) : for feature branches, we could run the full test suite only on the daily builds,

Re: [Tails-dev] Automated tests specification

2015-08-26 Thread bertagaz
Hi, On Wed, Aug 26, 2015 at 03:38:19PM +0200, anonym wrote: On 06/25/2015 11:41 AM, bertagaz wrote: Looks great! For the record, I looked at the spec as of commit e70f8e7. Thanks! I'm cheating, most of the work has already been done when we designed the autobuilds. :D # Facts Running

Re: [Tails-dev] Automated tests specification

2015-08-26 Thread anonym
On 08/26/2015 07:21 PM, bertagaz wrote: On Wed, Aug 26, 2015 at 02:00:25PM +0200, anonym wrote: On 07/01/2015 07:19 PM, intrigeri wrote: bertagaz wrote (25 Jun 2015 09:41:23 GMT) : for feature branches, we could run the full test suite only on the daily builds, and either only the automated

Re: [Tails-dev] Automated tests specification

2015-08-26 Thread kytv
On Wed, 26 Aug 2015 13:38:40 + (UTC) anonym ano...@riseup.net wrote: On 06/25/2015 11:41 AM, bertagaz wrote: I've prepared a blueprint to start this discussion and take notes of the decisions: https://tails.boum.org/blueprint/automated_builds_and_tests/automated_tests_specs/

Re: [Tails-dev] Automated tests specification

2015-08-26 Thread anonym
On 07/01/2015 07:19 PM, intrigeri wrote: bertagaz wrote (25 Jun 2015 09:41:23 GMT) : for feature branches, we could run the full test suite only on the daily builds, and either only the automated tests related to the branch on every git push, and/or a subset of the whole test suite. I'm not

Re: [Tails-dev] Automated tests specification

2015-08-26 Thread anonym
On 06/25/2015 11:41 AM, bertagaz wrote: I've prepared a blueprint to start this discussion and take notes of the decisions: https://tails.boum.org/blueprint/automated_builds_and_tests/automated_tests_specs/ Looks great! For the record, I looked at the spec as of commit e70f8e7. # Facts

Re: [Tails-dev] Automated tests specification

2015-08-23 Thread bertagaz
Hola, On Wed, Aug 19, 2015 at 08:58:32PM +0200, bertagaz wrote: So please contributors, take some time reading and commenting on this blueprint. Most of it is a copy of the automated builds one, as it is the next step in the chain and the previous one defined already most of our design.

Re: [Tails-dev] Automated tests specification

2015-08-19 Thread bertagaz
Hi, On Thu, Jul 09, 2015 at 04:52:12PM +0200, intrigeri wrote: bertagaz wrote (09 Jul 2015 13:28:23 GMT) : On Wed, Jul 01, 2015 at 07:19:04PM +0200, intrigeri wrote: bertagaz wrote (25 Jun 2015 09:41:23 GMT) : Indeed I find it too vague so I've rephrased this paragraph (414e4f3), and

Re: [Tails-dev] Automated tests specification

2015-07-09 Thread intrigeri
Hi, bertagaz wrote (09 Jul 2015 13:28:23 GMT) : On Wed, Jul 01, 2015 at 07:19:04PM +0200, intrigeri wrote: bertagaz wrote (25 Jun 2015 09:41:23 GMT) : I've tried to sum this up in a 'current proposal' subsection. Seen this, yay. And pushed a few commits on top. It also needs to know what

Re: [Tails-dev] Automated tests specification

2015-07-09 Thread bertagaz
Hi Thanks for your answer! On Wed, Jul 01, 2015 at 07:19:04PM +0200, intrigeri wrote: bertagaz wrote (25 Jun 2015 09:41:23 GMT) : I've prepared a blueprint to start this discussion and take notes of the decisions:

Re: [Tails-dev] Automated tests specification

2015-07-01 Thread intrigeri
Hi, bertagaz wrote (25 Jun 2015 09:41:23 GMT) : I've prepared a blueprint to start this discussion and take notes of the decisions: https://tails.boum.org/blueprint/automated_builds_and_tests/automated_tests_specs/ Great work! I've pushed a few minor changes, and a more important one

[Tails-dev] Automated tests specification

2015-06-25 Thread bertagaz
Hi, As the automated builds are soon going to be put in production (yes, they are), it's time to start the second round of discussion about the next coming Tails CI feature: automated testing of these automated build ISOs. Yay! I've prepared a blueprint to start this discussion and take notes