Hi Patrick,
thanks for picking this up.
From a high level perspective this is very simple. I try to sum it up
to help find an existing framework, which I tried several times but
failed.
When ignoring distributing binaries or blobs and only focussing on
test reports you have something similar to a review application.
We need
a) User management and authentication
b) Users being able to add new "products". In our case that's a
mainboard variant with a specific hardware configuration (memory,
hard-disk, PCIe cards, ...).
Every product has a custom set of properties you want to evaluate
and for which you can send in a test report. That could be
- boots OS x
- device Y detected and properly configured
- register Z locked and secure boot is possible
c) For every product in the database authenticated users could then
push status reports (reviews) which supply a test result for the
properties defined earlier.
The status-report would be submitted for a specific coreboot
version and build config.
You could 'add', 'list', 'sort', 'filter', 'delete' reports for
"products" as well.
Regards,
Patrick Rudolph
B.Sc. Electrical Engineering
System Firmware Developer
9elements GmbH, Kortumstraße 19-21, 44787 Bochum, Germany
Email: patrick.rudo...@9elements.com
Phone: +49 234 68 94 188
Sitz der Gesellschaft: Bochum
Handelsregister: Amtsgericht Bochum, HRB 17519
Geschäftsführung: Sebastian Deutsch, Eray Basar
Datenschutzhinweise nach Art. 13 DSGVO
Patrick Rudolph
B.Sc. Electrical Engineering
System Firmware Developer
9elements GmbH, Kortumstraße 19-21, 44787 Bochum, Germany
Email: patrick.rudo...@9elements.com
Phone: +49 234 68 94 188
Sitz der Gesellschaft: Bochum
Handelsregister: Amtsgericht Bochum, HRB 17519
Geschäftsführung: Sebastian Deutsch, Eray Basar
Datenschutzhinweise nach Art. 13 DSGVO
On Wed, Jun 16, 2021 at 10:35 PM Piotr Król <piotr.k...@3mdeb.com> wrote:
>
>
>
> On 6/16/21 8:46 PM, Patrick Georgi via coreboot wrote:
> > Hi everybody,
>
> Hi Patrick,
> thank you for bringing this topic to ml.
>
> >
> > There has been some talk recently in a smaller group where coreboot
> > needs to improve the most in public perception, and how to get there.
> >
> > Consensus has been that we're doing a pretty bad job at promoting all
> > the hardware that we support in each coreboot version.
> >
> > There's board-status which I started _years_ ago in the hope that
> > somebody picks up the slack, but everybody has been busy, myself
> > included. By now the collected information of the last 7.5 years is
> > compiled into a 12MB HTML file that takes ages to render on moderate
> > hardware (beware: https://www.coreboot.org/status/board-status.html
> > <https://www.coreboot.org/status/board-status.html>), and the process to
> > collect that data is mostly manual using pretty poor tooling. (Most of
> > the links on that page don't even work anymore (which I'll fix) due to
> > gitweb/cgit/gitiles changes on review.coreboot.org
> > <http://review.coreboot.org> (and I only noticed by chance now).)
> >
> > Meanwhile, there are several parties that boot test the hardware they
> > care about regularly, with (often internal) information about how well
> > coreboot does there.
>
> On of this parties is 3mdeb and we publish regression 150 test results
> for v4.0.x and mainline on 6 platforms:
> https://docs.google.com/spreadsheets/d/1_uRhVo9eYeZONnelymonYp444zYHT_Q_qmJEJ8_XqJc/edit?usp=sharing
>
> >
> > We can't expect all those existing systems to converge into a single
> > testing framework, but we could make it a single test result reporting
> > framework.
>
> This topic was discussed many times on various conferences and OSFW
> Slack. I believe contest aim to be that framework, please correct me if
> I'm misinterpreted something from OSFC'20.
>
> Missing things from perspective of sending test reports from 3mdeb
> validation infrastructure is REST API definition that can receive
> required board status data.
>
> For basic support maybe Qubes OS-like HCL would be good enough?
> https://www.qubes-os.org/doc/hcl/
>
> Please note Qubes do not force people to do git commits as in case of
> old board status, what lowers the barrier for reporting.
> https://github.com/QubesOS/qubes-hcl
>
> > To this end, I invite people interested in that topic to chime in on
> > this email thread so that we can discuss what we could do to provide a
> > common place with information about which coreboot versions are bootable
> > on which boards in a way that makes sense for everybody: users who are
> > interested in such data as well as testers that already collect it but
> > have no way to publish it.
>
> As it was mentioned on coreboot leadership validation system most
> probably will need tweaks/modifications/improvements of build system.
>
> Some future ideas that may came from build system could be:
> 1. coreboot.org to host build results for given defconfig and make it
> accessible to regular users - of course this would be vanilla coreboot
> since we agreed that in most cases production builds are different from
> what we have on coreboot.org. This lowers the barrier since regular
> users do not have to compile coreboot by themselves. Having confirmed,
> working binaries for given configuration would be huge win and step
> towards "stable releases", which I advertised in other discussions.
> 2. Firmware binaries could be delivered by fwupd/LVFS infrastructure
> This would largely help to reach more end users since by simple switch
> in fwupd they would be able to seamlessly deliver alternative firmware
> to their devices including updates.
>
> Best Regards,
> --
> Piotr Król
> Embedded Systems Consultant
> GPG: B2EE71E967AA9E4C
> https://3mdeb.com | @3mdeb_com
>
> _______________________________________________
> coreboot mailing list -- coreboot@coreboot.org
> To unsubscribe send an email to coreboot-le...@coreboot.org
_______________________________________________
coreboot mailing list -- coreboot@coreboot.org
To unsubscribe send an email to coreboot-le...@coreboot.org
