Re: [OE-Core][RFC][PATCH 3/3] testimage: implement test artifacts retriever for failing tests

2023-06-07 Thread Mikko Rapeli
Hi, On Wed, Jun 07, 2023 at 09:36:07AM +0200, Alexis Lothoré wrote: > Hi Mikko, sorry for late reply, and thanks for the additional feedback > > On 6/2/23 15:07, Mikko Rapeli wrote: > > Hi, > > > > These changes are an improvement, but based on my experience in product > > test automation, > >

Re: [OE-Core][RFC][PATCH 3/3] testimage: implement test artifacts retriever for failing tests

2023-06-07 Thread Alexis Lothoré via lists . openembedded . org
Hi Mikko, sorry for late reply, and thanks for the additional feedback On 6/2/23 15:07, Mikko Rapeli wrote: > Hi, > > These changes are an improvement, but based on my experience in product test > automation, > instead of collecting logs after testing is completed, it is better to > capture

Re: [OE-Core][RFC][PATCH 3/3] testimage: implement test artifacts retriever for failing tests

2023-06-02 Thread Mikko Rapeli
Hi, These changes are an improvement, but based on my experience in product test automation, instead of collecting logs after testing is completed, it is better to capture logs from the target device while tests are being executed. Serial console, systemd journal etc logs can be captured as

Re: [OE-Core][RFC][PATCH 3/3] testimage: implement test artifacts retriever for failing tests

2023-06-02 Thread Alexander Kanavin
On Fri, 2 Jun 2023 at 14:14, Alexis Lothoré wrote: > I wondered about this possibility while adding this variable, and at this > time I > thought it would encode to much "CI logic" in testimage class. But since you > raise the suggestion, I can update to pass directly the file list, which could

Re: [OE-Core][RFC][PATCH 3/3] testimage: implement test artifacts retriever for failing tests

2023-06-02 Thread Alexis Lothoré via lists . openembedded . org
Hello Alexander, Mikko, On 6/2/23 13:52, Mikko Rapeli wrote: > Hi, > > On Fri, Jun 02, 2023 at 01:46:03PM +0200, Alexander Kanavin wrote: >> On Fri, 2 Jun 2023 at 11:50, Alexis Lothoré via lists.openembedded.org >> wrote: >>> +# When any test fails, if path to an artifacts configuration

Re: [OE-Core][RFC][PATCH 3/3] testimage: implement test artifacts retriever for failing tests

2023-06-02 Thread Mikko Rapeli
Hi, On Fri, Jun 02, 2023 at 01:46:03PM +0200, Alexander Kanavin wrote: > On Fri, 2 Jun 2023 at 11:50, Alexis Lothoré via lists.openembedded.org > wrote: > > +# When any test fails, if path to an artifacts configuration (listing > > +# files/directories to retrieve on target) has been provided,

Re: [OE-Core][RFC][PATCH 3/3] testimage: implement test artifacts retriever for failing tests

2023-06-02 Thread Alexander Kanavin
On Fri, 2 Jun 2023 at 11:50, Alexis Lothoré via lists.openembedded.org wrote: > +# When any test fails, if path to an artifacts configuration (listing > +# files/directories to retrieve on target) has been provided, all listed > +# elements will be downloaded from target before shutting it down >

[OE-Core][RFC][PATCH 3/3] testimage: implement test artifacts retriever for failing tests

2023-06-02 Thread Alexis Lothoré via lists . openembedded . org
Add a basic artifacts retrievers in testimage class which: - triggers when at least one runtime test fails - reads a list of files to retrieve from ARTIFACTS_LIST_PATH - retrieve those files over scp thanks to existing ssh class - store those files in an "artifacts" directory in "tmp/log/oeqa/"