Re: [gentoo-user] Upgrading old kernel
On 4/15/20 10:59 PM, Thomas Mueller wrote: On 4/15/20 1:40 PM, Andreas Stiasny wrote: On 15.04.20 17:50, Rich Freeman wrote: Jumping from 3.18 you're somewhat more likely to run into issues - your biggest headache though will be dealing with the 30,000 prompts you get from make oldconfig and making sure you set all the new options correctly. That's why I use make olddefconfig in such a case. This takes all the old config values and uses the default for the new ones. If you know that you need one or more of the new config options you can fine tune them afterwards with make menuconfig. Andreas james responded: Ah. never used olddefconfig, I'll give it a spin. That raises the question, what if you have no kernel config, as may be the case if you are going to Gentoo for the first time, or are cross-compiling from FreeBSD or NetBSD? I have tried with OpenADK (www.openadk.org), which got as far as successfully building cross-gcc some of the time, but never succeeded at building the kernel. Is defconfig the best starting point? One would want to maximize the probability of success building the kernel while retaining a functional system that would support vital hardware including ethernet, wi-fi, hard drives and USB, and I would need to be able to read a NetBSD or FreeBSD file system (UFS/FFSv1 or 2). I use GPT, so there are no traditional now-deprecated BSD disklabels that Linux would not recognize. If I just start with menuconfig, I could miss some vital parts. OpenADK started with a minimal kernel config, maybe it was too minimal? I have successfully compiled kernels and userlands on FreeBSD and NetBSD (no menuconfig, defconfig, etc; kernel configs start with a GENERIC config). NetBSD kernel config is much longer than FreeBSD kernel config but is dwarfed by Linux kernel config. Tom OK, time to spill 'the beans'. OpenADK does not look like a kernel building tool. Booting a minimal state-machine for an embedded device, starts at the bottom of the code blocks. Building a linux kernel, that runs on the bottom of processors, guaged by resources and capabilities has always been a 'pita' that is nothing but duress. As you down the tree of what micro-processors can do, and the limited (kernel/system) resources, limited instruction sets, etc, etc that need is not common and you are best off following a well worn path. Folks that do not deeply understand the lack or limited (uP) resources and the subsequent limitied options available, need to get into a good, university program or go write assemble code on uPs for a few years. In essence, that sort of approach is a giant waste of time. I.E. follow a well worn path and learn to code in C and assembler. Executive, minimal OS and such, written in forth or other such languages are shear folly. C and Assembler, for find something else to do with your time, wisely. However, that said, integrating certain processor family trees into what other, more sophisticated 64 bit arm projects are doing, particularly with low level codes, is a wise idea. Pick your battles wisely. Caveat Emptor! Back to my thread:: An older, existing system is very rich in unique work and codes, at least for me, so I keep old image-systems, around for decades. A Gentoo packrat, as I can quote from very smart people back to 2004, when necessary. Their words were and are true, but, let's focus on virgin/noob (kernel-centric) systems issues. Fast Forward. I use the install disk from CloverOS which does a wonderful job of auto* for recent kernels and many packages. However it is not portage_raw or the myriad of other places to download and install or hack ebuilds; not necessarily of the Gentoo-approved feedstock. ymmv. CloverOS will give you a clean, new kernel, but lacks a window system (I do not use anything big, regardless of system resources) that is functional but not robust, imho. But 10 (15 max) minutes for a gentoo install is just freaking awesome, btrfs and a new kernel 5.* kernel. I have not tried to just copy over a kernel and associated file, but that is on the list as I have (3) identical AMD systems, 64bit, with AMD video cards and 32 G of DDR3 memory. But I shall just try to copy over a kernel derived from the CloverOS gentoo to an indentical hardware system running a version 3.18 james-derived-and-build linux kernel, just to see what happens. So a hybrid technique to rapidly test pre-built kernels, in an automated fashion, then going back and duplicating the same kernel-builds-tests from a kernel-gentoo-source-package, would be and attractive experiment to me. One off (which is what we do as a gentoo collective of hacks) kernels with the build-test-repeat cycle seems like an arcane semantic for a collective of experts (or fledgling gentoo-soon-to-be-experts)in this day and age (strictly of my opine). Where I'm going, managing a collective of hundreds or thousands of gentoo
Re: [gentoo-user] Upgrading old kernel
On 4/15/20 1:40 PM, Andreas Stiasny wrote: > On 15.04.20 17:50, Rich Freeman wrote: >> Jumping from >> 3.18 you're somewhat more likely to run into issues - your biggest >> headache though will be dealing with the 30,000 prompts you get from >> make oldconfig and making sure you set all the new options correctly. > That's why I use make olddefconfig in such a case. This takes all the > old config values and uses the default for the new ones. If you know > that you need one or more of the new config options you can fine tune > them afterwards with make menuconfig. > Andreas james responded: > Ah. never used olddefconfig, I'll give it a spin. That raises the question, what if you have no kernel config, as may be the case if you are going to Gentoo for the first time, or are cross-compiling from FreeBSD or NetBSD? I have tried with OpenADK (www.openadk.org), which got as far as successfully building cross-gcc some of the time, but never succeeded at building the kernel. Is defconfig the best starting point? One would want to maximize the probability of success building the kernel while retaining a functional system that would support vital hardware including ethernet, wi-fi, hard drives and USB, and I would need to be able to read a NetBSD or FreeBSD file system (UFS/FFSv1 or 2). I use GPT, so there are no traditional now-deprecated BSD disklabels that Linux would not recognize. If I just start with menuconfig, I could miss some vital parts. OpenADK started with a minimal kernel config, maybe it was too minimal? I have successfully compiled kernels and userlands on FreeBSD and NetBSD (no menuconfig, defconfig, etc; kernel configs start with a GENERIC config). NetBSD kernel config is much longer than FreeBSD kernel config but is dwarfed by Linux kernel config. Tom
Re: [gentoo-user] Upgrading old kernel
On Wed, Apr 15, 2020 at 2:31 PM james wrote: > > On 4/15/20 11:40 AM, Rich Freeman wrote: > > > I personally use the latest longterm, but not until it has been out > > for a few months. Mainly this is because I use zfs and don't want to > > deal with what versions of the one are compatible with what versions > > of the other. > > Yep, for the main system, but using btrfs with redundant drives. I'd > like zfs, but not certain about it's future being open, open-source, > etc. btrfs has bee great, for what I have done recently. > So, a few comments here: First, I used to use btrfs and I'd say it is just as important to stick with a longterm using btrfs because that project has a terrible history of introducing regressions in new kernel releases. If I were using btrfs I'm not sure I'd even go with 5.4 over 4.19 as it has only been around a few months and I'd be concerned they haven't worked out all the btrfs bugs yet. Now, I haven't used btrfs recently, so maybe things have gotten much better, but I'm skeptical on that front. I've had to do complete btrfs reinstalls more than once from backups, and this was on btrfs raid1 only. I REALLY like the feature set and design/etc of btrfs and think it definitely could be the future of linux mainstream storage, but for whatever reason QA has been a big problem and it has taken way longer than I expected to mature. It was btrfs QA problems that drove me to pay such close attention to what kernel series I was running. That is why I've mainly moved to zfs as my main general-purpose filesystem on hosts where restoring from backup isn't about popping an SDcard out of a Pi and flashing a couple GB backup image onto it. I'm not entirely happy with some of the limitations of zfs and of course it not being in-mainline is a huge hassle. There really is no risk of it not being FOSS - it is FOSS and of course it always will be as is the case with anything FOSS. Whether anybody is contributing to it in 10 years is another matter, but it isn't like the license has an expiry date on it. The #3 openzfs contributor is a Gentoo dev. I suspect the main risk to zfs is that btrfs finally gets its quality level up sufficiently that people switch over, which would be great. Either that or zfs gets sloppy with QA and people abandon it, which would be terrible, but probably unlikely at this point. For larger-scale storage I'm using Lizardfs and greatly admire Ceph as well in this space. MooseFS is another option (which Lizardfs is a fork of). These distributed filesystems are generally more flexible than zfs and give you redundancy above the host level. Right now the bulk of my storage is on lizardfs with the lizardfs chunkservers being implemented on top of zfs. That gives me the data security benefits of zfs but without the inflexibility, since I don't pool drives so I'm not limited by the ability to add/remove/etc drives from zfs pools. That said, vdev removal has become a thing in v0.8 and perhaps we'll see increased flexibility in the future. Overall zfs and btrfs are actually converging somewhat, just from different target audiences. IMO with the growing importance of distributed filesystems I think that the main niche for zfs and btrfs will be as a general-purpose filesystem similar to ext4 but with additional flexibility (volume management) and robustness (raid/checksums/etc). Once you get bigger than a few drives Ceph will become the gold standard for storage, or at least that is the leading technology right now. Lizardfs is more of a ghetto Ceph that doesn't require dozens of GB of RAM per server. If you haven't been upgrading kernels you may have just missed all the fun of btrfs regressions over recent years. :) In any case unless things have changed a lot I'd seriously consider longterms and then carefully checking for regressions before doing upgrades between major versions. -- Rich
Re: [gentoo-user] Upgrading old kernel
On 4/15/20 11:40 AM, Rich Freeman wrote: On Wed, Apr 15, 2020 at 11:27 AM james wrote: It works fabulously, but it is time to upgrade, as most codes dependent on old software, have been migrated. So should I skip to a version 5 kernel? If so which one? I usually run hundreds of testing packages so maybe make the new system all testing? If you're more of the mindset of stability over features (as seems to be the case) then I'd stick with a longterm kernel. That means years of updates that basically shouldn't require anything more than running make oldconfig to deal with. Once in a VERY rare while a new option shows up. Traditionally yes, but not going forward. About 1/2 are on (going to be) the latest and I'll probably just default to every package being the latest testing, github or whatever version. You should be updating your kernel regularly to address security issues and other regressions. If you stay within the same major.minor series you shouldn't be getting anything other than bugfixes. Agreed, but most of my systems rarely have a route to the internet or are mostly not connected to any ethernet, continuously. I personally use the latest longterm, but not until it has been out for a few months. Mainly this is because I use zfs and don't want to deal with what versions of the one are compatible with what versions of the other. Yep, for the main system, but using btrfs with redundant drives. I'd like zfs, but not certain about it's future being open, open-source, etc. btrfs has bee great, for what I have done recently. Right now I'm on the 4.19 longterm, and I'm getting to the point where I'm contemplating switching to the 5.4 longterm. If I were in your shoes i'd be looking at 5.4 unless there is a reason not to. 5.4 sounds good. If you're asking how to actually compile/install/etc a kernel just follow the docs, but you should be doing this regularly. Jumping from 3.18 you're somewhat more likely to run into issues - your biggest headache though will be dealing with the 30,000 prompts you get from make oldconfig and making sure you set all the new options correctly. You won't get that problem going between two patch-level releases (eg 5.4.31 -> 5.4.32). Agreed. I was bad sick, off and on for 3 years. Rare blood sugar. 80% protein diet fixed it all. NO medications, no sugar very few slow carbs, finally. So, basically my mind was 80% erased. Good thing I kept notes and a myriad of sporadic 'howto docs'. Kernel hacking was void for 3 years. Now I feel GREAT and have many gentoo ambitions, 5G and embedded centric stuff; but also a mail and a web server, with very tight security. DNS primaries on little, ram intensive arm boards, are pretty sweet when combined with cloudflare's free, secure dns. Thank for all the help/ideas, James
Re: [gentoo-user] Upgrading old kernel
On 4/15/20 1:40 PM, Andreas Stiasny wrote: On 15.04.20 17:50, Rich Freeman wrote: Jumping from 3.18 you're somewhat more likely to run into issues - your biggest headache though will be dealing with the 30,000 prompts you get from make oldconfig and making sure you set all the new options correctly. That's why I use make olddefconfig in such a case. This takes all the old config values and uses the default for the new ones. If you know that you need one or more of the new config options you can fine tune them afterwards with make menuconfig. Andreas Ah. never used olddefconfig, I'll give it a spin.
Re: [gentoo-user] Upgrading old kernel
On 15.04.20 17:50, Rich Freeman wrote: Jumping from 3.18 you're somewhat more likely to run into issues - your biggest headache though will be dealing with the 30,000 prompts you get from make oldconfig and making sure you set all the new options correctly. That's why I use make olddefconfig in such a case. This takes all the old config values and uses the default for the new ones. If you know that you need one or more of the new config options you can fine tune them afterwards with make menuconfig. Andreas
[gentoo-user] Upgrading old kernel
Hello, So I have a gentoo system, with a 3.18.25 kernel. It works fabulously, but it is time to upgrade, as most codes dependent on old software, have been migrated. So should I skip to a version 5 kernel? If so which one? I usually run hundreds of testing packages so maybe make the new system all testing? Note: Not interested in using genkernel; prefer discrete, old-fashions method of understandable steps; but perhaps scripted... I run many old gentoo systems, from embedded hacks on 32 bit micros to AMD centric workstations (old but reliable). So, I really would like a system/guide that is general, but where I can add details for the various hardware/architectures/ issues. It seems like for 18 years, it's been a 'whatever reboots' incoherent system and I'm tire of it. I need structure and consistency, as I might not upgrade/install any system for months/years and then I need to do a dozen or more, not necessarily of the same hardware. AMD and ARM are what they are mostly, but other variants are in my collective. Hardware is like kids, it's hard for me to let go; ymmv. BTW, a guide that is BTRFS centric, would really be cool All discussion/suggestions are welcome, including comments on the links below. Is this the best guide to follow? https://www.kernel.org/doc/html/latest/ or this one? https://wiki.gentoo.org/wiki/Kernel/Upgrade https://wiki.gentoo.org/wiki/Kernel and this I found this noob_simplified_doc: https://blog.khmersite.net/2019/01/upgrade-kernel-on-gentoo/ Really looking for input for one generalized guide that can have options for various hardware combos. curiously, James
Re: [gentoo-user] Upgrading old kernel
On Wed, Apr 15, 2020 at 11:27 AM james wrote: > > It works fabulously, but it is time to upgrade, as most codes dependent > on old software, have been migrated. > > So should I skip to a version 5 kernel? > If so which one? I usually run hundreds of testing packages so maybe > make the new system all testing? If you're more of the mindset of stability over features (as seems to be the case) then I'd stick with a longterm kernel. That means years of updates that basically shouldn't require anything more than running make oldconfig to deal with. Once in a VERY rare while a new option shows up. You should be updating your kernel regularly to address security issues and other regressions. If you stay within the same major.minor series you shouldn't be getting anything other than bugfixes. I personally use the latest longterm, but not until it has been out for a few months. Mainly this is because I use zfs and don't want to deal with what versions of the one are compatible with what versions of the other. Right now I'm on the 4.19 longterm, and I'm getting to the point where I'm contemplating switching to the 5.4 longterm. If I were in your shoes i'd be looking at 5.4 unless there is a reason not to. If you're asking how to actually compile/install/etc a kernel just follow the docs, but you should be doing this regularly. Jumping from 3.18 you're somewhat more likely to run into issues - your biggest headache though will be dealing with the 30,000 prompts you get from make oldconfig and making sure you set all the new options correctly. You won't get that problem going between two patch-level releases (eg 5.4.31 -> 5.4.32). -- Rich
