On 06/28/2018 07:39 AM, Quentin Schulz wrote:
Hi Tom,
On Wed, Jun 27, 2018 at 11:23:38AM -0400, Tom Rini wrote:
On Wed, Jun 27, 2018 at 09:52:50AM +0200, Quentin Schulz wrote:
Hi Tom,
On Tue, Jun 26, 2018 at 10:44:59AM -0400, Tom Rini wrote:
On Tue, Jun 26, 2018 at 02:41:40PM +0200, Quentin Schulz wrote:
Hi all,
On Wed, Jun 13, 2018 at 01:02:06PM -0600, Stephen Warren wrote:
On 06/13/2018 12:53 PM, Quentin Schulz wrote:
Hi Tom,
On Wed, Jun 13, 2018 at 11:43:32AM -0400, Tom Rini wrote:
On Mon, Jun 04, 2018 at 11:47:30AM +0200, Quentin Schulz wrote:
This tests that the importing of an environment with a specified
whitelist works as intended.
If there are variables passed as parameter to the env import command,
those only should be imported in the current environment.
For each variable passed as parameter, if
- foo is bar in current env and bar2 in exported env, after importing
exported env, foo shall be bar2,
- foo does not exist in current env and foo is bar2 in exported env,
after importing exported env, foo shall be bar2,
- foo is bar in current env and does not exist in exported env (but is
passed as parameter), after importing exported env, foo shall be empty,
Any variable not passed as parameter should be left untouched.
Two other tests are made to test that size cannot be '-' if the checksum
protection is enabled.
Signed-off-by: Quentin Schulz <quentin.sch...@bootlin.com>
Reviewed-by: Simon Glass <s...@chromium.org>
Reviewed-by: Stephen Warren <swar...@nvidia.com>
This seems to not be working?
https://travis-ci.org/trini/u-boot/jobs/391504525
I just rebased on top of v2018.07-rc1, ran
make mrproper
./test/py/test.py --bd sandbox --build
and the tests run fine ...
Most likely the failure is due to the test relying on some feature that
isn't enabled on the board being tested (emulated via qemu); you'll need to
add something like the following to indicate which feature the test relies
upon:
@pytest.mark.buildconfigspec('cmd_echo')
OK, I've added the dependency on cmd_importenv and cmd_exportenv, but
that does not make it work any better.
I added my U-Boot repo to Travis and ran the tests. Here is the output
of the job: https://travis-ci.org/QSchulz/u-boot/
Specifically, you have:
https://travis-ci.org/QSchulz/u-boot/jobs/396742661
https://travis-ci.org/QSchulz/u-boot/jobs/396742668
https://travis-ci.org/QSchulz/u-boot/jobs/396742669
https://travis-ci.org/QSchulz/u-boot/jobs/396742670
https://travis-ci.org/QSchulz/u-boot/jobs/396742671
I've dumped the RAM after the `env export` and it looks pretty much
empty compared to what I could see with sandbox tests.
Since all the other tests work, I'm not sure I actually introduced a
regression or if it just never worked. I'll run tests without my patches
that do a `env export` followed by a dump of the memory, a reset of the
environement and a `env import` to see where we stand right now.
It's possible you've exposed a latent bug here. What stood out to me at
the time was that it looked like we were using 0x0 for the address and
that's quite possibly an invalid location to be using on these
platforms.
Yes, looked weird to me as well but can't tell if that's a legit RAM
address. Stephen says other tests interacting with RAM works on those
configs so that should be working.
So, I tested without my patches for whitelisting and it does not work
for the same configs:
https://travis-ci.org/QSchulz/u-boot/jobs/396898590
https://travis-ci.org/QSchulz/u-boot/jobs/396898597
https://travis-ci.org/QSchulz/u-boot/jobs/396898598
https://travis-ci.org/QSchulz/u-boot/jobs/396898599
I'm not introducing a regression with my patch. Do we know if env import
and export is actually supported by those configs? How could we fix
those? Let me know if I can help.
Maybe the tests need to depend on something else too then, that's not
enabled on these platforms?
On another subject, I'm worried by these failing jobs:
https://travis-ci.org/QSchulz/u-boot/jobs/396898591
https://travis-ci.org/QSchulz/u-boot/jobs/396898592
Does this mean that we don't have a clean environnement for each and
every test? (env import complains when I'm trying to import an
environnement with ethXaddr in it, so I forcibly removed them from the
environnement before exporting).
Right, tests are run serially and don't reset the board for each, iirc.
Is there a reason behind this behaviour?
We should fix this somewhere in the future, the tests are basically
meaningless otherwise. If we don't start from scratch for each and every
test (I'm not even talking about test series (e.g. all env tests) but
each test that is called), we basically test that this test works when
all the other tests before have behaved as they did. At best, you get
false positives, at worst you don't catch bugs.
This is deliberate for a couple of reasons:
1) In order to catch general stability problems such as memory
corruption, we do as much as possible in a single "instance" (boot) of
U-Boot, so that any such problem has a maximal chance of negatively
affecting future tests. Yes, this can cause false positives as you
describe, but in my experience that has happened rarely in practice; I
vaguely recall one instance of a test working when run after another
test (something in DFU on one of the targets I use?) and one instance of
a test failing only when run after another test (some unit test in
sandbox IIRC), so it balanced out nicely.
2) It's quicker not to reboot the system for each test, which reduces
test time, which allows for more testing (more test cases).
_______________________________________________
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot
