CI Wrapper tests

2019-05-21 Thread Bruno Wolff III
I am trying to get a wrapped test script to work for squashfs-tools and 
the artifacts don't appear to be properly saved. I think I might be 
misunderstanding the wrapper example. Is the idea really that the test 
should run ansible with the basic role referring to the wrapper? I 
have been trying to run the wrapper as the test and it runs the shell 
script, but the test results don't end up where the CI can find them 
and the test fails.
I also don't see a way for the wrapper to ask for extra packages other 
than to run dnf in the script.


There is stuff to look at in the pipeline pages if you want to see what I 
tried to do.

___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org


Re: [Xen-devel] Criteria / validation proposal: drop Xen

2019-05-21 Thread Adam Williamson
On Mon, 2019-05-13 at 16:29 -0600, Lars Kurth wrote:
> Hi all,
> 
> I am going to step in here with my hat as Xen Project community
> manager. We had a discussion about this issue as part of last week's
> community call. I CC'ed a number of stake-holders, which probably
> should have been on the thread such as ITL (which builds QubesOS
> on top of Fedora) and Michael A Young (the Xen package manager).
> 
> First of all apologies that this issue has lingered so long. Key
> members of the community were not aware of the issues raised in
> this thread, otherwise we would have acted earlier. With this in
> mind, please in future raise issues with me, on xen-devel@,
> committers@ or the Xen-Fedora package manager. The Xen Community
> would like to see Fedora running as guest: in fact it would be
> somewhat odd if there was a Xen-Dom0 package and guest support
> didn't work. And there are some downstreams such as QubesOS,
> which depend on this support.

Thanks for stepping in. Of course we always want as much stuff as
possible to work, but that does not mean we block the release on it. We
certainly want Fedora to work as a guest on VMWare, VirtualBox and
Parallels too; we don't block the release on any of those either...

> > Well, I mean, every few *days* a compose gets nominated for validation
> > testing, and a mail is sent to test-announce. Just check your test-
> > announce archives for mails with "nominated for testing" in their
> > subject lines, and you'll see dozens. Is this not sufficient
> > notification?
> 
> We discussed this at the community call and came to the conclusion that
> we can run regular tests of Fedora RC's as part of our OSSTEST
> infrastructure. Ian Jackson volunteered to implement this, but there
> are some questions on
> a) The installer (which we can handle ourselves)
> b) When we would trigger a test - aka is there some trigger other than the
> c) How would results best be reported back to Fedora
> 
> Apologies, I am not very familiar with how the Fedora Test group works.
> Is there some documentation which would help integrate what you to with
> the test system of another open source project?

b) you can use fedmsg / fedora-messaging:
https://github.com/fedora-infra/fedmsg
https://github.com/fedora-infra/fedora-messaging
A message is emitted every time a compose attempt finishes (on the fedmsg
topic 'org.fedoraproject.prod.pungi.compose.status.change': see
https://apps.fedoraproject.org/datagrepper/raw?topic=org.fedoraproject.prod.pungi.compose.status.change
for a log of past messages). What you will want to do is listen for
completed Branched and Rawhide composes and run tests whenever one
completes successfully. This is already exactly what we do to schedule
openQA tests; you can crib from the openQA test scheduler:
https://pagure.io/fedora-qa/fedora_openqa
particularly the fedmsg consumer:
https://pagure.io/fedora-qa/fedora_openqa/blob/master/f/fedora_openqa/consumer.py

c) ideally it would be good to report to both resultsdb and to the
wiki. Again, we already do this for openQA, and you can crib from the
code there:
https://pagure.io/fedora-qa/fedora_openqa/blob/master/f/fedora_openqa/report.py
reporting to ResultsDB might be tricky due to authentication issues,
I'm not sure if we ever put the openID auth stuff into production. For
wiki reporting you will either have to auth manually every so often or
ask Fedora infra for a special token that doesn't expire (this is what
we do for the openQA results).
Reporting to ResultsDB you do through resultsdb_api -
https://pagure.io/taskotron/resultsdb_api - and optionally you can use
my resultsdb_conventions -
https://pagure.io/taskotron/resultsdb_conventions - which makes it
somewhat easier (IMO anyway) and will make your results consistent with
those from openQA and Autocloud. Reporting to the wiki you can do
through my crazy python-wikitcms library -
https://pagure.io/fedora-qa/python-wikitcms . Again, fedora_openqa does
all this for openQA results, so you can crib from that. Let me know if
you have trouble with this.

> > > > from Oracle. On that basis, I'm proposing we remove this Final
> > > > criterion:
> > > 
> > > s/Oracle/Xen Project/ I believe?
> > 
> > Perhaps, it's just that it always seemed to be you doing the testing,
> > so they got a bit conflated :)
> 
> Can we come to some arrangement, by which such issues get communicated
> to the Xen Project earlier? Aka me, xen-devel@ or committers@

It would be nice if you could ensure someone from Xen is actually
watching the Fedora lists, if working in Fedora is a target for Xen. We
*could* try and CC stuff all the time, but imagine if we tried to do
that for everybody. But yes, for future conversations of this nature
I'll try and remember to include those lists.

> > > > "The release must boot successfully as Xen DomU with releases providing
> > > > a functional, supported Xen Dom0 and widely used cloud providers
> > > > utilizing Xen."
> > > > 
> > > > and change the 'milest

Re: [Xen-devel] Criteria / validation proposal: drop Xen

2019-05-21 Thread Kamil Paral
On Mon, May 20, 2019 at 7:45 PM Lars Kurth  wrote:

> @Adam and Fedora Testing & QA:
> any views on my proposal?
> Regards
> Lars
>

Hi Lars,
thanks for your reply. Adam was on a long vacation and he's probably the
most qualified person to reply to you, sorry for not telling you sooner.
Adam is now back, so he should hopefully join the conversation shortly.

Cheers,
Kamil
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org