Re: [OE-core] [PATCH v14] Rust Oe-Selftest implementation

[Richard Purdie]

Hi Sundeep,

On Tue, 2023-06-06 at 02:47 -0700, sundeep.kokko...@eng.windriver.com
wrote:
> Thank you for review, testing and confirming that oe-selftest is
> passed. We've worked on the regressions caused with the patch and
> fixed the issue.
> The rust builds (bitbake rust/nativesdk-rust) were not part of our
> regular oe-selftest so we've not tested those earlier. From now we
> will make these tests are  part of our regular testing.
>  
> With latest sources there are a few new test case failures observed
> and we are working on it.
> Also, we see there is a rust 1.70.0 upgrade patch in mailing list. We
> will send the V15 after rebasing to it.

That explains why we had a misunderstanding in the test results. The
v15 sounds good, thanks!

Cheers,

Richard

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182423): 
https://lists.openembedded.org/g/openembedded-core/message/182423
Mute This Topic: https://lists.openembedded.org/mt/99216990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v14] Rust Oe-Selftest implementation

[sundeep . kokkonda]

Hello Richard,

Thank you for review, testing and confirming that oe-selftest is passed. We've 
worked on the regressions caused with the patch and fixed the issue.
The rust builds (bitbake rust/nativesdk-rust) were not part of our regular 
oe-selftest so we've not tested those earlier. From now we will make these 
tests are  part of our regular testing.

With latest sources there are a few new test case failures observed and we are 
working on it.
Also, we see there is a rust 1.70.0 upgrade patch in mailing list. We will send 
the V15 after rebasing to it.

Thank you,
Sundeep K.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182418): 
https://lists.openembedded.org/g/openembedded-core/message/182418
Mute This Topic: https://lists.openembedded.org/mt/99216990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v14] Rust Oe-Selftest implementation

[Shinde, Yash via lists.openembedded.org]

Hi Richard,

Thanks for the confirmation.

Regards,
Yash

From: Richard Purdie
Sent: 02 June 2023 17:02
To: Shinde, Yash; Kokkonda, 
Sundeep; 
openembedded-core@lists.openembedded.org
Cc: MacLeod, Randy; Kallapa, 
Umesh; Gowda, 
Naveen; Moodalappa, 
Shivaprasad
Subject: Re: [PATCH v14] Rust Oe-Selftest implementation

CAUTION: This email comes from a non Wind River email account!
Do not click links or open attachments unless you recognize the sender and know 
the content is safe.

Hi Yash,

On Fri, 2023-06-02 at 07:25 +, Shinde, Yash wrote:
>
> As discussed in the meeting, I am sharing the steps followed and host 
> configurations used for the testing-
>
> Steps followed for testing-
>
> 1. Clone a fresh poky from $ git clone git://git.yoctoproject.org/poky
> 2. source the build environment setup script
> 3. add the meta-selftest layer in conf/bblayers.conf
> 4. add SANITY_TESTED_DISTROS="" to local.conf
> 5. Apply the testing patch changes-
>a. Add  meta/lib/oeqa/selftest/cases/rust.py file containing testing code
>b. Add meta/recipes-devtools/rust/files/rust-oe-selftest.patch to exclude 
> failing cases.
>c. Add rust-oe-selftest.patch in 
> meta/recipes-devtools/rust/rust-source.inc to apply on rust sources.
>d. Add "do_test_compile ()" for testing rust sources in 
> meta/recipes-devtools/rust/rust_1.69.0.bb
>e. Run the oe-selftest via "oe-selftest -r 
> rust.RustSelfTestBase.test_check" command.

As I promised, I did try these steps and you are correct that they are
successful. However, if I try:

"bitbake rust"

I see the failure here:

https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/7128/steps/12/logs/stdio

likewise, I suspect "bitbake nativesdk-rust" will also fail with the
other failure mentioned there.

This does not happen unless your patch is applied so it is a
regression. There are likely other failures however this one is pretty
fundamental so we should start by fixing that.

I did document my exact steps as follows:

$ git clone git://git.yoctoproject.org/poky poky-rusttest
$ cd poky-rusttest/
$ source oe-init-build-env
$ echo 'BBLAYERS += "/media/build/poky-rusttest/meta-selftest"' >>
conf/bblayers.conf
$ echo 'SANITY_TESTED_DISTROS = ""' >> conf/local.conf
$ echo 'DL_DIR = "/media/sources/"' >> conf/local.conf
$ wget
https://lore.kernel.org/openembedded-core/20230530105125.445622-1-yashinde...@gmail.com/raw
$ git am -s raw
$ oe-selftest -r rust.RustSelfTestBase.test_check
$ bitbake rust nativesdk-rust -k

where the only real difference to your steps was to set DL_DIR to my
local sources cache to avoid downloading things needlessly.


Cheers,

Richard


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182305): 
https://lists.openembedded.org/g/openembedded-core/message/182305
Mute This Topic: https://lists.openembedded.org/mt/99216990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v14] Rust Oe-Selftest implementation

[Richard Purdie]

Hi Yash,

On Fri, 2023-06-02 at 07:25 +, Shinde, Yash wrote:
> 
> As discussed in the meeting, I am sharing the steps followed and host 
> configurations used for the testing-
>  
> Steps followed for testing-
>  
> 1. Clone a fresh poky from $ git clone git://git.yoctoproject.org/poky
> 2. source the build environment setup script
> 3. add the meta-selftest layer in conf/bblayers.conf
> 4. add SANITY_TESTED_DISTROS="" to local.conf
> 5. Apply the testing patch changes-
>    a. Add  meta/lib/oeqa/selftest/cases/rust.py file containing testing code
>    b. Add meta/recipes-devtools/rust/files/rust-oe-selftest.patch to exclude 
> failing cases.
>    c. Add rust-oe-selftest.patch in 
> meta/recipes-devtools/rust/rust-source.inc to apply on rust sources.
>    d. Add "do_test_compile ()" for testing rust sources in 
> meta/recipes-devtools/rust/rust_1.69.0.bb
>    e. Run the oe-selftest via "oe-selftest -r 
> rust.RustSelfTestBase.test_check" command.

As I promised, I did try these steps and you are correct that they are
successful. However, if I try:

"bitbake rust"

I see the failure here:

https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/7128/steps/12/logs/stdio

likewise, I suspect "bitbake nativesdk-rust" will also fail with the
other failure mentioned there.

This does not happen unless your patch is applied so it is a
regression. There are likely other failures however this one is pretty
fundamental so we should start by fixing that.

I did document my exact steps as follows:

$ git clone git://git.yoctoproject.org/poky poky-rusttest
$ cd poky-rusttest/
$ source oe-init-build-env
$ echo 'BBLAYERS += "/media/build/poky-rusttest/meta-selftest"' >>
conf/bblayers.conf 
$ echo 'SANITY_TESTED_DISTROS = ""' >> conf/local.conf 
$ echo 'DL_DIR = "/media/sources/"' >> conf/local.conf 
$ wget
https://lore.kernel.org/openembedded-core/20230530105125.445622-1-yashinde...@gmail.com/raw
$ git am -s raw
$ oe-selftest -r rust.RustSelfTestBase.test_check
$ bitbake rust nativesdk-rust -k

where the only real difference to your steps was to set DL_DIR to my
local sources cache to avoid downloading things needlessly.


Cheers,

Richard


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182304): 
https://lists.openembedded.org/g/openembedded-core/message/182304
Mute This Topic: https://lists.openembedded.org/mt/99216990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v14] Rust Oe-Selftest implementation

[Sundeep KOKKONDA via lists.openembedded.org]

Hello Richard,

We've (re)tested and reviewed the results of this patch on our local setup and 
those are "PASSED".
We will discuss further with Randy and get back to community.



Thanks,
Sundeep K.



From: Richard Purdie 
Sent: 30 May 2023 19:29
To: Yash Shinde ; 
openembedded-core@lists.openembedded.org 

Cc: MacLeod, Randy ; Kallapa, Umesh 
; Gowda, Naveen ; 
Kokkonda, Sundeep ; Moodalappa, Shivaprasad 
; Shinde, Yash 
Subject: Re: [PATCH v14] Rust Oe-Selftest implementation

CAUTION: This email comes from a non Wind River email account!
Do not click links or open attachments unless you recognize the sender and know 
the content is safe.

On Tue, 2023-05-30 at 16:21 +0530, Yash Shinde wrote:
> The patch implements Rust testing framework similar to other selftest,
> specifically the gcc selftest in OE. It uses the client and server
> based method to test the binaries for cross-target on the image.
> The test framework is a wrapper around the Rust build system as ./x.py
> test. It tests many functionalities of Rust distribution like tools,
> documentation, libraries, packages, tools, Cargo, Crater etc.
> Please refer the following link for detailed description of Rust
> testing:-
> https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-tests
>
> To support the rust tests in oe-core, the following functions were
> added:-
> setup_cargo_environment(): Build bootstrap and some early stage tools.
> do_rust_setup_snapshot(): Install the snapshot version of rust binaries.
> do_configure(): To generate config.toml
> do_compile(): To build "remote-test-server" for qemu target image.
>
> Approximate Number of Tests Run in the Rust Testsuite :- 18000
> Approximate Number of Tests that FAIL in bitbake environment :- 100-150
> Normally majority of the testcases are present in major folder "test/"
> It contributes to more than 80% of the testcases present in Rust test
> framework. These tests pass as expected on any Rust versions without
> much fuss. The tests that fail are of less important and contribute to
> less than 2% of the total testcases. These minor tests are observed to
> work on some versions and fail on others. They have to be added, ignored
> or excluded for different versions as per the behavior.
> These tests have been ignored or excluded in the Rust selftest
> environment to generate success of completing the testsuite.
>
> These tests work in parallel mode even in the skipped test mode as
> expected. Although the patch to disable tests is large, it is very simple
> in that it only disables tests. When updating to a newer version of Rust,
> the patch can usually be ported in a day.
>
> Tested for X86, X86-64, ARM, ARM64 and MIPS64 on CentOS release 6.10
>
> Signed-off-by: pgowda 
> Signed-off-by: Vinay Kumar 
> Signed-off-by: Yash Shinde 
> ---
>  meta/lib/oeqa/selftest/cases/rust.py  |   86 +
>  .../rust/files/rust-oe-selftest.patch | 2120 +
>  meta/recipes-devtools/rust/rust-source.inc|1 +
>  meta/recipes-devtools/rust/rust_1.69.0.bb |7 +
>  4 files changed, 2214 insertions(+)
>  create mode 100644 meta/lib/oeqa/selftest/cases/rust.py
>  create mode 100644 meta/recipes-devtools/rust/files/rust-oe-selftest.patch

I did put this into a build for testing on the autobuilder. It hasn't
been running long and I can already see build failures such as:

https://autobuilder.yoctoproject.org/typhoon/#/builders/103/builds/6077
https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/7200
https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/7190
https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2923
https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/7200
https://autobuilder.yoctoproject.org/typhoon/#/builders/48/builds/7217
https://autobuilder.yoctoproject.org/typhoon/#/builders/104/builds/6080
https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/7146
https://autobuilder.yoctoproject.org/typhoon/#/builders/61/builds/7184

and there will likely be more.

I'm finding this rather frustrating and am not sure how this was
tested.

Cheers,

Richard


oe-selftest-results.log
Description: oe-selftest-results.log

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181995): 
https://lists.openembedded.org/g/openembedded-core/message/181995
Mute This Topic: https://lists.openembedded.org/mt/99216990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v14] Rust Oe-Selftest implementation

[Frederic Martinsons]

Le mar. 30 mai 2023, 23:20, Richard Purdie <
richard.pur...@linuxfoundation.org> a écrit :

> On Tue, 2023-05-30 at 14:28 -0400, Randy MacLeod wrote:
> > >
> > > I did put this into a build for testing on the autobuilder. It
> > > hasn't
> > > been running long and I can already see build failures such as:
> > >
> > >
> https://autobuilder.yoctoproject.org/typhoon/#/builders/103/builds/6077
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/7200
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/7190
> > >
> https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2923
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/7200
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/48/builds/7217
> > >
> https://autobuilder.yoctoproject.org/typhoon/#/builders/104/builds/6080
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/7146
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/61/builds/7184
> > >
> > > and there will likely be more.
> > >
> > > I'm finding this rather frustrating and am not sure how this was
> > > tested.
> >
> > That's probably my fault. I've been asking Yash to keep pushing and
> > suggesting that I could help him with testing but the bad news is
> > that we (Wind River) have the tail end of a product release cycle
> > happening now.
> >
> > I'd like to help out with this, but it'll have to wait for at least 2
> > months and that puts us close to the end of M3 so we might have to
> > wait for the beginning of the fail development cycle.
> >
> > Let me talk with Yashe and Sundeep and reply with info on what to
> > expect.
>
> Let me try a different request on the public list too. We're clearly
> struggling to get this patch over the line. I have a finite amount of
> time and haven't really been able to dive into this properly, I have
> tried to guide Yash and Sundeep on some of the rust issues as best I
> can but I'm struggling.
>
> Is anyone else able to help out with some of the rust issues? The two
> pressing ones are the ptests (this patch) and the reproducibility
> issue. If anyone is able to help look into these, help is welcome,
> please talk to Randy and I.
>
> Cheers,
>
> Richard
>
>
> Several weeks ago, I gave a shot and spent some time investigating the
reproducibility issue (on zvariant recipe) but I was kind of stuck and I
was taken by other work on my side.
I think I can spent more hours on this during June. But of course, any help
is welcome since I currently have no leads.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181990): 
https://lists.openembedded.org/g/openembedded-core/message/181990
Mute This Topic: https://lists.openembedded.org/mt/99216990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v14] Rust Oe-Selftest implementation

[Randy MacLeod via lists.openembedded.org]

On 2023-05-30 17:20, Richard Purdie wrote:

wait for the beginning of the fail development cycle.


Ugh, that's: FALL development cycle.

What an unfortunate typo!!

--
# Randy MacLeod
# Wind River Linux

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181967): 
https://lists.openembedded.org/g/openembedded-core/message/181967
Mute This Topic: https://lists.openembedded.org/mt/99216990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v14] Rust Oe-Selftest implementation

[Richard Purdie]

On Tue, 2023-05-30 at 14:28 -0400, Randy MacLeod wrote:
> > 
> > I did put this into a build for testing on the autobuilder. It
> > hasn't
> > been running long and I can already see build failures such as:
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/103/builds/6077
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/7200
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/7190
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2923
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/7200
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/48/builds/7217
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/104/builds/6080
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/7146
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/61/builds/7184
> > 
> > and there will likely be more.
> > 
> > I'm finding this rather frustrating and am not sure how this was
> > tested.
>  
> That's probably my fault. I've been asking Yash to keep pushing and
> suggesting that I could help him with testing but the bad news is
> that we (Wind River) have the tail end of a product release cycle
> happening now. 
>  
> I'd like to help out with this, but it'll have to wait for at least 2
> months and that puts us close to the end of M3 so we might have to
> wait for the beginning of the fail development cycle.
>  
> Let me talk with Yashe and Sundeep and reply with info on what to
> expect.

Let me try a different request on the public list too. We're clearly
struggling to get this patch over the line. I have a finite amount of
time and haven't really been able to dive into this properly, I have
tried to guide Yash and Sundeep on some of the rust issues as best I
can but I'm struggling.

Is anyone else able to help out with some of the rust issues? The two
pressing ones are the ptests (this patch) and the reproducibility
issue. If anyone is able to help look into these, help is welcome,
please talk to Randy and I.

Cheers,

Richard





-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181952): 
https://lists.openembedded.org/g/openembedded-core/message/181952
Mute This Topic: https://lists.openembedded.org/mt/99216990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v14] Rust Oe-Selftest implementation

[Alexandre Belloni via lists.openembedded.org]

Hello,

On 30/05/2023 14:28:24-0400, Randy MacLeod via lists.openembedded.org wrote:
> On 2023-05-30 11:05, Frédéric Martinsons wrote:
> > Oh okay, didn't try that personnaly.
> > the class is currently tied to cargo behavior so for performing rust
> > core tests, it will not be suitable (at least in its present form)
> 
> Well, that's good news in that we won't have to change our approach for v15
> !
> 
> 
> On 2023-05-30 09:59, Richard Purdie wrote:
> > I did put this into a build for testing on the autobuilder. It hasn't
> > been running long and I can already see build failures such as:
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/103/builds/6077
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/7200
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/7190
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2923
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/7200
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/48/builds/7217
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/104/builds/6080
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/7146
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/61/builds/7184
> > 
> > and there will likely be more.
> > 
> > I'm finding this rather frustrating and am not sure how this was
> > tested.
> 
> That's probably my fault. I've been asking Yash to keep pushing and
> suggesting that
> I could help him with testing but the bad news is that we (Wind River)
> have the tail end of a product release cycle happening now.
> 
> I'd like to help out with this, but it'll have to wait for at least 2 months
> and
> that puts us close to the end of M3 so we might have to wait for the
> beginning
> of the fail development cycle.
> 
> Let me talk with Yashe and Sundeep and reply with info on what to expect.

I can run builds on the autobuilders on free cycles as long as it is
clearly communicated that the patch has not been tested so it doesn't
impair other patch testing.



-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181951): 
https://lists.openembedded.org/g/openembedded-core/message/181951
Mute This Topic: https://lists.openembedded.org/mt/99216990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v14] Rust Oe-Selftest implementation

[Randy MacLeod via lists.openembedded.org]

On 2023-05-30 11:05, Frédéric Martinsons wrote:

Oh okay, didn't try that personnaly.
the class is currently tied to cargo behavior so for performing rust 
core tests, it will not be suitable (at least in its present form)


Well, that's good news in that we won't have to change our approach for 
v15 !



On 2023-05-30 09:59, Richard Purdie wrote:

I did put this into a build for testing on the autobuilder. It hasn't
been running long and I can already see build failures such as:

https://autobuilder.yoctoproject.org/typhoon/#/builders/103/builds/6077
https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/7200
https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/7190
https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2923
https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/7200
https://autobuilder.yoctoproject.org/typhoon/#/builders/48/builds/7217
https://autobuilder.yoctoproject.org/typhoon/#/builders/104/builds/6080
https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/7146
https://autobuilder.yoctoproject.org/typhoon/#/builders/61/builds/7184

and there will likely be more.

I'm finding this rather frustrating and am not sure how this was
tested.


That's probably my fault. I've been asking Yash to keep pushing and 
suggesting that

I could help him with testing but the bad news is that we (Wind River)
have the tail end of a product release cycle happening now.

I'd like to help out with this, but it'll have to wait for at least 2 
months and
that puts us close to the end of M3 so we might have to wait for the 
beginning

of the fail development cycle.

Let me talk with Yashe and Sundeep and reply with info on what to expect.

--
# Randy MacLeod
# Wind River Linux

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181941): 
https://lists.openembedded.org/g/openembedded-core/message/181941
Mute This Topic: https://lists.openembedded.org/mt/99216990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v14] Rust Oe-Selftest implementation

[Frederic Martinsons]

Oh okay, didn't try that personnaly.
the class is currently tied to cargo behavior so for performing rust core
tests, it will not be suitable (at least in its present form)

Le mar. 30 mai 2023, 16:44, Richard Purdie <
richard.pur...@linuxfoundation.org> a écrit :

> On Tue, 2023-05-30 at 16:35 +0200, Frédéric Martinsons wrote:
> > Beside the errors pointed by Richard, I'm wondering how it behave
> > with the ptest-cargo class? Does this class still useful with this
> > work? Is it some kind of complementary or did it concern a totally
> > different matter/use case ?
>
> As I understand it, the intent in this change was to enable the core
> rust tests to run as ptests.
>
> Could we use ptest-cargo for that instead for the core rust recipe? If
> so, has anyone tried it?
>
> Cheers,
>
> Richard
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181934): 
https://lists.openembedded.org/g/openembedded-core/message/181934
Mute This Topic: https://lists.openembedded.org/mt/99216990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v14] Rust Oe-Selftest implementation

[Richard Purdie]

On Tue, 2023-05-30 at 16:35 +0200, Frédéric Martinsons wrote:
> Beside the errors pointed by Richard, I'm wondering how it behave
> with the ptest-cargo class? Does this class still useful with this
> work? Is it some kind of complementary or did it concern a totally
> different matter/use case ? 

As I understand it, the intent in this change was to enable the core
rust tests to run as ptests.

Could we use ptest-cargo for that instead for the core rust recipe? If
so, has anyone tried it?

Cheers,

Richard

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181931): 
https://lists.openembedded.org/g/openembedded-core/message/181931
Mute This Topic: https://lists.openembedded.org/mt/99216990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v14] Rust Oe-Selftest implementation

[Frederic Martinsons]

Beside the errors pointed by Richard, I'm wondering how it behave with the
ptest-cargo class? Does this class still useful with this work? Is it some
kind of complementary or did it concern a totally different matter/use case
?

Le mar. 30 mai 2023, 15:59, Richard Purdie <
richard.pur...@linuxfoundation.org> a écrit :

> On Tue, 2023-05-30 at 16:21 +0530, Yash Shinde wrote:
> > The patch implements Rust testing framework similar to other selftest,
> > specifically the gcc selftest in OE. It uses the client and server
> > based method to test the binaries for cross-target on the image.
> > The test framework is a wrapper around the Rust build system as ./x.py
> > test. It tests many functionalities of Rust distribution like tools,
> > documentation, libraries, packages, tools, Cargo, Crater etc.
> > Please refer the following link for detailed description of Rust
> > testing:-
> > https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-tests
> >
> > To support the rust tests in oe-core, the following functions were
> > added:-
> > setup_cargo_environment(): Build bootstrap and some early stage tools.
> > do_rust_setup_snapshot(): Install the snapshot version of rust binaries.
> > do_configure(): To generate config.toml
> > do_compile(): To build "remote-test-server" for qemu target image.
> >
> > Approximate Number of Tests Run in the Rust Testsuite :- 18000
> > Approximate Number of Tests that FAIL in bitbake environment :- 100-150
> > Normally majority of the testcases are present in major folder "test/"
> > It contributes to more than 80% of the testcases present in Rust test
> > framework. These tests pass as expected on any Rust versions without
> > much fuss. The tests that fail are of less important and contribute to
> > less than 2% of the total testcases. These minor tests are observed to
> > work on some versions and fail on others. They have to be added, ignored
> > or excluded for different versions as per the behavior.
> > These tests have been ignored or excluded in the Rust selftest
> > environment to generate success of completing the testsuite.
> >
> > These tests work in parallel mode even in the skipped test mode as
> > expected. Although the patch to disable tests is large, it is very simple
> > in that it only disables tests. When updating to a newer version of Rust,
> > the patch can usually be ported in a day.
> >
> > Tested for X86, X86-64, ARM, ARM64 and MIPS64 on CentOS release 6.10
> >
> > Signed-off-by: pgowda 
> > Signed-off-by: Vinay Kumar 
> > Signed-off-by: Yash Shinde 
> > ---
> >  meta/lib/oeqa/selftest/cases/rust.py  |   86 +
> >  .../rust/files/rust-oe-selftest.patch | 2120 +
> >  meta/recipes-devtools/rust/rust-source.inc|1 +
> >  meta/recipes-devtools/rust/rust_1.69.0.bb |7 +
> >  4 files changed, 2214 insertions(+)
> >  create mode 100644 meta/lib/oeqa/selftest/cases/rust.py
> >  create mode 100644
> meta/recipes-devtools/rust/files/rust-oe-selftest.patch
>
> I did put this into a build for testing on the autobuilder. It hasn't
> been running long and I can already see build failures such as:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/103/builds/6077
> https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/7200
> https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/7190
> https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2923
> https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/7200
> https://autobuilder.yoctoproject.org/typhoon/#/builders/48/builds/7217
> https://autobuilder.yoctoproject.org/typhoon/#/builders/104/builds/6080
> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/7146
> https://autobuilder.yoctoproject.org/typhoon/#/builders/61/builds/7184
>
> and there will likely be more.
>
> I'm finding this rather frustrating and am not sure how this was
> tested.
>
> Cheers,
>
> Richard
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181928): 
https://lists.openembedded.org/g/openembedded-core/message/181928
Mute This Topic: https://lists.openembedded.org/mt/99216990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v14] Rust Oe-Selftest implementation

[Richard Purdie]

On Tue, 2023-05-30 at 16:21 +0530, Yash Shinde wrote:
> The patch implements Rust testing framework similar to other selftest,
> specifically the gcc selftest in OE. It uses the client and server
> based method to test the binaries for cross-target on the image.
> The test framework is a wrapper around the Rust build system as ./x.py
> test. It tests many functionalities of Rust distribution like tools,
> documentation, libraries, packages, tools, Cargo, Crater etc.
> Please refer the following link for detailed description of Rust
> testing:-
> https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-tests
> 
> To support the rust tests in oe-core, the following functions were
> added:-
> setup_cargo_environment(): Build bootstrap and some early stage tools.
> do_rust_setup_snapshot(): Install the snapshot version of rust binaries.
> do_configure(): To generate config.toml
> do_compile(): To build "remote-test-server" for qemu target image.
> 
> Approximate Number of Tests Run in the Rust Testsuite :- 18000
> Approximate Number of Tests that FAIL in bitbake environment :- 100-150
> Normally majority of the testcases are present in major folder "test/"
> It contributes to more than 80% of the testcases present in Rust test
> framework. These tests pass as expected on any Rust versions without
> much fuss. The tests that fail are of less important and contribute to
> less than 2% of the total testcases. These minor tests are observed to
> work on some versions and fail on others. They have to be added, ignored
> or excluded for different versions as per the behavior.
> These tests have been ignored or excluded in the Rust selftest
> environment to generate success of completing the testsuite.
> 
> These tests work in parallel mode even in the skipped test mode as
> expected. Although the patch to disable tests is large, it is very simple
> in that it only disables tests. When updating to a newer version of Rust,
> the patch can usually be ported in a day.
> 
> Tested for X86, X86-64, ARM, ARM64 and MIPS64 on CentOS release 6.10
> 
> Signed-off-by: pgowda 
> Signed-off-by: Vinay Kumar 
> Signed-off-by: Yash Shinde 
> ---
>  meta/lib/oeqa/selftest/cases/rust.py  |   86 +
>  .../rust/files/rust-oe-selftest.patch | 2120 +
>  meta/recipes-devtools/rust/rust-source.inc|1 +
>  meta/recipes-devtools/rust/rust_1.69.0.bb |7 +
>  4 files changed, 2214 insertions(+)
>  create mode 100644 meta/lib/oeqa/selftest/cases/rust.py
>  create mode 100644 meta/recipes-devtools/rust/files/rust-oe-selftest.patch

I did put this into a build for testing on the autobuilder. It hasn't
been running long and I can already see build failures such as:

https://autobuilder.yoctoproject.org/typhoon/#/builders/103/builds/6077
https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/7200
https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/7190
https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2923
https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/7200
https://autobuilder.yoctoproject.org/typhoon/#/builders/48/builds/7217
https://autobuilder.yoctoproject.org/typhoon/#/builders/104/builds/6080
https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/7146
https://autobuilder.yoctoproject.org/typhoon/#/builders/61/builds/7184

and there will likely be more.

I'm finding this rather frustrating and am not sure how this was
tested.

Cheers,

Richard

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181925): 
https://lists.openembedded.org/g/openembedded-core/message/181925
Mute This Topic: https://lists.openembedded.org/mt/99216990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-