On 31.03.2023 16:47, Daniel Golle wrote:
On Fri, Mar 31, 2023 at 03:52:47PM +0300, Arınç ÜNAL wrote:
On 31.03.2023 14:33, Daniel Golle wrote:
Hi!
On Fri, Mar 31, 2023 at 12:44:12PM +0300, Arınç ÜNAL wrote:
Hi all,
These are the ideas I've been thinking about for the future of OpenWrt for a
while. It looks complete enough to share it with all of you.
I'm willing to put a great deal of effort to get as much out-of-tree patches
on mainline Linux as possible.
You can make a comment on Notion or discuss it here, I'm wondering if the
ideas are feasible and how well it would benefit the people maintaining
OpenWrt.
https://arinc9.notion.site/OpenWrt-next-gen-ideas-6db745e7584b4823950291c96f2326bb
I will comment here, I don't have an account on Notion and it seems
to be required to be able to comment there.
defconfig for each device instead of config for each (sub)target.
Given that we support thousands of devices this will not only increase
the time needed to build a release or snapshot by several magnitudes,
but also make debugging **much** harder. As of now, all devices of a
subtarget are using the same kernel, hence e.g. symbol offsets in a
kernel stack dump match for all of them. To reproduce or investigate
a problem it's hence enough to have similar hardware, not necessarily
the exact same board. As we are already lacking testers and maintainers
for the relatively small amount of targets/subtargets, have a build for
each board would make things much worse...
Per-device builds would also be an invitation to downstream users to
introduce device-specific (kernel-)hacks. If you want that, better go
for OpenEmbedded.
We can modularize things more or even have more sub-targets if it's
really needed to save space.
The disadvantages outweight the advantages imho when it comes to having
a complete kernel build for each device.
Hmm, what about we enable the bare minimum of kernel options for a target,
which is already how it is, then select the rest as kernel modules (like on
the makefile of a target for each device) on the defconfig for each device?
So, in the end, it wouldn't be any different than selecting a kernel module
package from the OpenWrt SDK which, I believe, does not change the symbol
offsets in the kernel stack.
My reason for pushing for the use defconfigs is that anyone can build the
Linux kernel for their device, without needing OpenWrt. So the work for
adding support for a device would benefit far more people.
This is pretty much what we are currently doing.
The exception are network drivers to allow for failsafe mode to work
and provide SSH access **before** any modules are loaded.
Got it, network drivers should also be built into the kernel on the
defconfigs then.
Some Modest Virtualization Observations
How is this related? Virtualization (with OpenWrt being the guest)
matters on x86 which is usually not that space-constraint.
And maybe armvirt. If space is a problem for older x86 boards, let's
disable guest support in x86/legacy.
It was a broad example without any explanation. What caught my attention
there is the configuration of the kernel that causes problems, if I
understand correctly.
Contribute defconfigs and all the devicetrees on OpenWrt to Linux.
For devicetrees this would of course be desirable, but also implies a
lot of work and discussions. If you are up to get it started (ie. setup
a tree to collect cleaned-up and ready to submit dts), I think it would
be worth the effort, at least for more recent targets/SoCs.
I've been meaning to do this for the mt7621 SoC devices for months. The main
roadblock is that some drivers are out-of-tree, like the NAND flash so it
makes no sense to have them defined on the devicetree. Getting the
out-of-tree patches on mainline Linux is another step so it'll happen
eventually.
Hm, I thought that Weijie had sent the mt7621-nand driver also upstream,
but I haven't been following the process...
I haven't checked for a while too, maybe it did get in. Then there's the
mac incrementing on the devicetree which doesn't exist on mainline Linux.
I'll get this started with my Linux fork on GitHub.
Very nice, I will join in there.
I'm leaving the link here for future reference.
https://github.com/arinc9/linux
Regarding defconfigs I don't think we need an individual defconfig for
each board. The problem here is also that OpenWrt currently has a layered
approach (generic->target->subtarget) approach while Linux itself has
a flat approach, and using that would result in a lot of duplication,
which would in turn make keeping all those defconfigs up-to-date quite
a lot of work...
Not really, once you make a defconfig for a device, the only case for it to
change in the future is if a kernel option was renamed, which is very rare.
Anyway, even in that case, there are known ways to update them by bulk.
https://lore.kernel.org/kernel-janitors/20220929090645.1389-1-lukas.bulw...@gmail.com/
I agree for "established" platforms for which drivers are already
upstream and not many changes are still expected to happen.
Yet, a defconfig **for each board** seems like overkill to me, esp.
because 90%+ will not be able to boot the resulting image generated
by a vanilla kernel build in any meaningful way.
Having defconfig for each sub-target would be agreeable, though I must
say that I do like our layerd approach to defconfig.
Maybe we should pick single representative **and resourceful** boards
and try upstreaming defconfig for those (in case it didn't already happen).
However, not using the generic defconfig anyway is only relevant for
rather resource-constraint boards (as otherwise you'd just use, let's say
multi_v7_defconfig or the like). Are you sure upstream is interested in
being flooded with hundreds of defconfigs, most of them for very similar
devices?
I'm not sure, there's Thomas who maintains MIPS. There's a lot of ARM
maintainers. I'll have to talk it through with them.
The main issue I see that forces me to make defconfig per device is
because every device will slightly need different kernel modules.
It's basically taking the kernel modules from here and put it to the
defconfig for the device.
https://github.com/openwrt/openwrt/blob/master/target/linux/ramips/image/mt7621.mk#L137
Either submit all out-of-tree patches on OpenWrt to Linux or get rid
of them and find a better solution for what the unacceptable patch
does.
This would of course be great, but especially for legacy devices it may
not be possible in many cases. Think of all the devices stuck on
swconfig, just to name one example... Think of all the completely
broken vendor bootloaders which require hacks (mangeling kernel cmdline
and such) and cannot easily be replaced...
Those can stay until eventually the support for them will be dropped on
newer OpenWrt versions. I believe there are a lot of out-of-tree patches
that are not for old devices and can be on mainline Linux.
In terms of hardware support you are probably right.
When it comes to vendor bootloaders which require mangeling kernel
bootargs the kernel folks have always argued that users should just
change the bootloader environment -- however, this requires a serial
console attached and makes reverting to stock firmware harder.
Makes sense. It makes sense to maintain this part on OpenWrt.
Another example is selection of the rootfs. Kernel folks argue that
we should use an initramfs for that, however, we try to avoid the
overhead of using an initramfs with it's own userland just for that.
To resolve this situation at least for future boards (ie. with
post-2020 versions of U-Boot) I've put quite some effort into adding
native support for mapping the filesystem sub-image(s) of a uImage.FIT
image as a block device in Linux:
https://github.com/dangowrt/linux/commit/8a70f52fddd518d1d7093a12874ca3a5de86fab1
I've already had quite a lot of debate about that, currently I'm waiting
for add/remove/size-change notifications for in-kernel block device users
being added to the kernel, as that would be required for a really clean
implementation.
This is very nice. Please CC me when there's anything new. This was one
of the things I was planning to figure out. I think the current way,
even with a FIT image, is to put the location of the filesystem device
on the bootargs so the kernel can find it once the SPI/NAND driver
probes the flash?
Most other patches in target/linux/generic/pending-5.* should be sutiable
for upstream inclusion. Patches in target/linux/generic/hack-5.* are more
tragic cases...
Apart from the technical debate the issue here is of course also of
organizational nature. While not accepting new patches unless they are
already accepted upstream is a matter of policy, upstreaming all the
existing patches is a lot of work, and the question arises *who* is
going to volunteer do that...
This is what I'm planning to spend my time on for the next couple of years.
Bugfix backporting should happen only after it's accepted to Linux.
The patch must be identical to the commit on Linux.
The wording here might be a bit too strict to support our existing
mess, but I generally agree. So I'd say 'should' instead of 'must', but
otherwise agree.
Feature backporting should be done only if it's thoroughly tested.
... and testing often happens in the OpenWrt tree. So it's a bit of
a chicken-egg problem, as often developers don't even have all the
different hardware needed for testing. But generally I agree.
A way to ease testing *before* pushing to openwrt.git or posting to
upstream mailing lists would be to have snapshot builds also for
developers' staging trees.
Kernel Solution
Make a mode menu.
Filesystem only.
So which kernel headers should be used to build e.g. libc and netlink
users?
In a way it is also currently possible to build generic images for
most architectures using the armvirt, malta and x86 targets. Of course,
also in this case a kernel is being built.
Make a kernel selection menu where the user can choose to feed the
kernel directory of their own or use the longterm one defined on the
OpenWrt SDK. Add this as a suboption to the full image mode.
What about CONFIG_EXTERNAL_KERNEL_TREE and friends...?
I've never tried this but looks like it may need a bit of extra options.
No, I'm using this on a daily base to use linux-next and friends together
with OpenWrt userland. For some devices you may need to add some
downstream patches, but e.g. recent MediaTek stuff works out-of-the-box.
I couldn't get netifd to work properly on 6.2 when I was just booting
the OpenWrt filesystem as initramfs. I'll try this option though, thanks.
No patches must be applied, kernel module packages must not be compiled,
etc.
Kernel module packages work fine, just the version will be wrong in case
of an external kernel tree being used.
If I remember correctly, with 5.17, some netfilter kernel option was
changed which caused the compilation of a kernel module package to fail.
Having the option to disable them altogether would address this type of
edge cases.
Arınç
_______________________________________________
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
