On 31.03.2023 18:36, Daniel Golle wrote:
On Fri, Mar 31, 2023 at 05:35:22PM +0300, Arınç ÜNAL wrote:
On 31.03.2023 16:47, Daniel Golle wrote:
On Fri, Mar 31, 2023 at 03:52:47PM +0300, Arınç ÜNAL wrote:
On 31.03.2023 14:33, Daniel Golle wrote:
Hi!
On Fri, Mar 31, 2023 at 12:44:12PM +0300, Arınç ÜNAL wrote:
Hi all,
These are the ideas I've been thinking about for the future of OpenWrt for a
while. It looks complete enough to share it with all of you.
I'm willing to put a great deal of effort to get as much out-of-tree patches
on mainline Linux as possible.
You can make a comment on Notion or discuss it here, I'm wondering if the
ideas are feasible and how well it would benefit the people maintaining
OpenWrt.
https://arinc9.notion.site/OpenWrt-next-gen-ideas-6db745e7584b4823950291c96f2326bb
I will comment here, I don't have an account on Notion and it seems
to be required to be able to comment there.
defconfig for each device instead of config for each (sub)target.
Given that we support thousands of devices this will not only increase
the time needed to build a release or snapshot by several magnitudes,
but also make debugging **much** harder. As of now, all devices of a
subtarget are using the same kernel, hence e.g. symbol offsets in a
kernel stack dump match for all of them. To reproduce or investigate
a problem it's hence enough to have similar hardware, not necessarily
the exact same board. As we are already lacking testers and maintainers
for the relatively small amount of targets/subtargets, have a build for
each board would make things much worse...
Per-device builds would also be an invitation to downstream users to
introduce device-specific (kernel-)hacks. If you want that, better go
for OpenEmbedded.
We can modularize things more or even have more sub-targets if it's
really needed to save space.
The disadvantages outweight the advantages imho when it comes to having
a complete kernel build for each device.
Hmm, what about we enable the bare minimum of kernel options for a target,
which is already how it is, then select the rest as kernel modules (like on
the makefile of a target for each device) on the defconfig for each device?
So, in the end, it wouldn't be any different than selecting a kernel module
package from the OpenWrt SDK which, I believe, does not change the symbol
offsets in the kernel stack.
My reason for pushing for the use defconfigs is that anyone can build the
Linux kernel for their device, without needing OpenWrt. So the work for
adding support for a device would benefit far more people.
This is pretty much what we are currently doing.
The exception are network drivers to allow for failsafe mode to work
and provide SSH access **before** any modules are loaded.
Got it, network drivers should also be built into the kernel on the
defconfigs then.
Some Modest Virtualization Observations
How is this related? Virtualization (with OpenWrt being the guest)
matters on x86 which is usually not that space-constraint.
And maybe armvirt. If space is a problem for older x86 boards, let's
disable guest support in x86/legacy.
It was a broad example without any explanation. What caught my attention
there is the configuration of the kernel that causes problems, if I
understand correctly.
Contribute defconfigs and all the devicetrees on OpenWrt to Linux.
For devicetrees this would of course be desirable, but also implies a
lot of work and discussions. If you are up to get it started (ie. setup
a tree to collect cleaned-up and ready to submit dts), I think it would
be worth the effort, at least for more recent targets/SoCs.
I've been meaning to do this for the mt7621 SoC devices for months. The main
roadblock is that some drivers are out-of-tree, like the NAND flash so it
makes no sense to have them defined on the devicetree. Getting the
out-of-tree patches on mainline Linux is another step so it'll happen
eventually.
Hm, I thought that Weijie had sent the mt7621-nand driver also upstream,
but I haven't been following the process...
I haven't checked for a while too, maybe it did get in. Then there's the mac
incrementing on the devicetree which doesn't exist on mainline Linux.
I think Rafal has been taking care of that lately, but I might be
wrong.
label-mac also is a downstream use of DTS specific to OpenWrt which
didn't yet make it upstream.
I'll get this started with my Linux fork on GitHub.
Very nice, I will join in there.
I'm leaving the link here for future reference.
https://github.com/arinc9/linux
Thank you!
Regarding defconfigs I don't think we need an individual defconfig for
each board. The problem here is also that OpenWrt currently has a layered
approach (generic->target->subtarget) approach while Linux itself has
a flat approach, and using that would result in a lot of duplication,
which would in turn make keeping all those defconfigs up-to-date quite
a lot of work...
Not really, once you make a defconfig for a device, the only case for it to
change in the future is if a kernel option was renamed, which is very rare.
Anyway, even in that case, there are known ways to update them by bulk.
https://lore.kernel.org/kernel-janitors/20220929090645.1389-1-lukas.bulw...@gmail.com/
I agree for "established" platforms for which drivers are already
upstream and not many changes are still expected to happen.
Yet, a defconfig **for each board** seems like overkill to me, esp.
because 90%+ will not be able to boot the resulting image generated
by a vanilla kernel build in any meaningful way.
Having defconfig for each sub-target would be agreeable, though I must
say that I do like our layerd approach to defconfig.
Maybe we should pick single representative **and resourceful** boards
and try upstreaming defconfig for those (in case it didn't already happen).
However, not using the generic defconfig anyway is only relevant for
rather resource-constraint boards (as otherwise you'd just use, let's say
multi_v7_defconfig or the like). Are you sure upstream is interested in
being flooded with hundreds of defconfigs, most of them for very similar
devices?
I'm not sure, there's Thomas who maintains MIPS. There's a lot of ARM
maintainers. I'll have to talk it through with them.
The main issue I see that forces me to make defconfig per device is because
every device will slightly need different kernel modules.
It's basically taking the kernel modules from here and put it to the
defconfig for the device.
https://github.com/openwrt/openwrt/blob/master/target/linux/ramips/image/mt7621.mk#L137
Taking MT7621 as the example I believe that the main differences are
mostly the selection of wifi drivers, USB support and occasionally AHCI.
You really think it makes sense to provide 200+ defconfigs just to
cover the (limited) number of combinations?
Wouldn't a handful or even a single defconfig be sufficient?
Yeah, I agree. A defconfig with the most basic stuff and network drivers
as built-in, the rest as kernel modules per (sub)target should be
enough. I'll get started with the mt7621 subtarget.
The "the most basic stuff and network drivers as built-in" part is
already covered with current configs on OpenWrt, if I understand
correctly. For the kernel modules, I guess I'll just collect them from
the OpenWrt makefile.
Either submit all out-of-tree patches on OpenWrt to Linux or get rid
of them and find a better solution for what the unacceptable patch
does.
This would of course be great, but especially for legacy devices it may
not be possible in many cases. Think of all the devices stuck on
swconfig, just to name one example... Think of all the completely
broken vendor bootloaders which require hacks (mangeling kernel cmdline
and such) and cannot easily be replaced...
Those can stay until eventually the support for them will be dropped on
newer OpenWrt versions. I believe there are a lot of out-of-tree patches
that are not for old devices and can be on mainline Linux.
In terms of hardware support you are probably right.
When it comes to vendor bootloaders which require mangeling kernel
bootargs the kernel folks have always argued that users should just
change the bootloader environment -- however, this requires a serial
console attached and makes reverting to stock firmware harder.
Makes sense. It makes sense to maintain this part on OpenWrt.
Another example is selection of the rootfs. Kernel folks argue that
we should use an initramfs for that, however, we try to avoid the
overhead of using an initramfs with it's own userland just for that.
To resolve this situation at least for future boards (ie. with
post-2020 versions of U-Boot) I've put quite some effort into adding
native support for mapping the filesystem sub-image(s) of a uImage.FIT
image as a block device in Linux:
https://github.com/dangowrt/linux/commit/8a70f52fddd518d1d7093a12874ca3a5de86fab1
I've already had quite a lot of debate about that, currently I'm waiting
for add/remove/size-change notifications for in-kernel block device users
being added to the kernel, as that would be required for a really clean
implementation.
This is very nice. Please CC me when there's anything new. This was one of
the things I was planning to figure out. I think the current way, even with
a FIT image, is to put the location of the filesystem device on the bootargs
so the kernel can find it once the SPI/NAND driver probes the flash?
Yes, the current way is very platform-specific and storage-type specific.
With recent devboards supporting several boot devices on the same board
I felt the need to have one unified method which can work on all of them,
hence I came up with the uImage.FIT partition parser. The partition parser,
however, has been rejected upstream and I was told to instead implement
this as a tiny block driver.
At least for devices with more or less recent U-Boot this is imho a
very good option.
This will be very useful for companies that use Buildroot, mainline
U-Boot and Linux on their own embedded devices. This will be THE
solution for where I work. I was already in the middle of writing
somewhat of a guide for this whole process.
Most other patches in target/linux/generic/pending-5.* should be sutiable
for upstream inclusion. Patches in target/linux/generic/hack-5.* are more
tragic cases...
Apart from the technical debate the issue here is of course also of
organizational nature. While not accepting new patches unless they are
already accepted upstream is a matter of policy, upstreaming all the
existing patches is a lot of work, and the question arises *who* is
going to volunteer do that...
This is what I'm planning to spend my time on for the next couple of years.
That's very good to hear! Let me know if you have any questions
regarding existing patches, for most I should be able to provide you
with some background information.
Thanks a lot. I'm planning to present this whole plan on Battlemesh v15.
This conversation helped shape it in my mind a lot.
Arınç
_______________________________________________
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
