Hello Heinrich,
Am 22.06.2019 um 21:12 schrieb Heinrich Schuchardt:
On 6/22/19 8:15 PM, Simon Glass wrote:
Hi,
On Sat, 22 Jun 2019 at 16:10, Andreas Färber <afaer...@suse.de> wrote:
Hi Simon,
Am 22.06.19 um 16:55 schrieb Simon Glass:
I'd like to better understand the benefits of the 3-month timeline.
It takes time to learn about a release, package and build it, test it on
various hardware, investigate and report errors, wait for feedback and
fixes, rinse and repeat with the next -rc. Many people don't do this as
their main job.
If we shorten the release cycle, newer boards will get out faster (which
is good) but the overall quality of boards not actively worked on
(because they were working good enough before) will decay, which is bad.
The only way to counteract that would be to automatically test on real
hardware rather than just building, and doing that for all these masses
of boards seems unrealistic.
Here I think you are talking about distributions. But why not just
take every second release?
I have certain had the experience of getting a board our of the
cupboard and finding that the latest U-Boot doesn't work, nor the one
before, nor the three before that.
Are we actually seeing an improvement in regressions? I feel that
testing is the only way to get that.
Perhaps we should select a small subset of boards which do get tested,
and actually have custodians build/test on those for every rc?
What I have been doing before all my recent pull requests is to boot
both an arm32 (Orange Pi) and and an aarch64 (Pine A64 LTS) board via
bootefi and GRUB. To make this easier I am using a Raspberry with a
relay board and a Tizen SD-Wire card (https://wiki.tizen.org/SDWire)
controlling the system under test,
cf https://pbs.twimg.com/media/D5ugi3iX4AAh1bn.jpg:large
What would be needed is scripts to automate the testing including all
the Python tests.
It would make sense to have such test automation for all of our
architectures similar to what Kernel CI (https://kernelci.org/) does.
Yes ... my dream is also we have a lot of boards, on which we can test
automagically. My approach was tbot, with which I made weekly automated
commandline tests (u-boot and linux), see example results from my old
tbot version:
http://xeidos.ddns.net/tests/test_db_auslesen.php#987
(Intentionally link to latest good test, as I found no time yet to
refresh my testsetup to new tbot, see later).
Above webpage is created through a tbot generator, which fills a
mysql database and the webpage is a simple php script. I also speculated
to write a generator to connect to kernel-ci ... but time is the missing
resource.
For the above example tbot and the webserver run on a raspberry pi,
also the raspberry pi is used as "Lab Host" which controlls the board
under test, so cheap hardware costs. It is easy to adapt tbot to your
hardware setup, see [2]
But tbot was a hack as my python skills are not the greatest, so
luckily Harald (added to cc) found time to rewrite it completly from
scratch (many thanks!), see [1] and [2]
Ok, missing here and there functionality from the old version, but
it is much more cleaner ... may you take a look at it, also it is
Open Source, feel free to help and improve tbot!
I use tbot for my daily work, as tbot has also an interactive mode [3],
with which you can use tbot for powering on your board and connect
to for example the u-boot command line). No need anymore to know where
your board is and how to power it on or how to connect to the console.
(Background I mostly have no real physical access to hardware I work
on)
So you can work while developing with tbot, and at the end you have
immediately an automated setup you can integrate into your CI. As
tbot is a commandline tool, this is mostly easy to do.
Also it is possible to call pytest framework [4] from u-boot,
and of course you can call this out of tbot [5].
bye,
Heiko
[1] https://github.com/Rahix/tbot
[2] https://rahix.de/tbot/
[3] https://rahix.de/tbot/getting-started.html#interactive
[4]
http://git.denx.de/?p=u-boot.git;a=blob;f=test/README;h=4bc9ca3a6ae9de0e3ed6ff420c5edfe6027ad2c6;hb=77f6e2dd0551d8a825bab391a1bd6b838874bcd4
[5] result from pytest framework called from tbot
http://xeidos.ddns.net/tbot/id_985/test-log.html
Regards
Heinrich
_______________________________________________
U-Boot-Custodians mailing list
u-boot-custodi...@lists.denx.de
https://lists.denx.de/listinfo/u-boot-custodians
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-52 Fax: +49-8142-66989-80 Email: h...@denx.de
_______________________________________________
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot
