Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
"H. Peter Anvin" <[EMAIL PROTECTED]> writes: > Eric W. Biederman wrote: >> >> With ELF we get a single file format that works on multiple >> architectures and for multiple OS-s, with well understood rules. This >> allows for a broader audience who can review and maintain the code. >> In addition to the real possibility of multi-architecture or multi-os >> bootloaders. >> > > Well understood MAYBE (although the "virtual" versus "physical" address stuff > in > ELF is clearly misspecified in the extreme), but they are clearly not > sufficient > for presenting a well-defined entry condition. So it's not really anywhere > near > as straightforward as you make it sound. About the entry condition I completely agree. The fact that the generic tools almost do what you need and then fall short is also problematic. However it still appears a reasonable approach to consider. Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Eric W. Biederman wrote: With ELF we get a single file format that works on multiple architectures and for multiple OS-s, with well understood rules. This allows for a broader audience who can review and maintain the code. In addition to the real possibility of multi-architecture or multi-os bootloaders. Well understood MAYBE (although the "virtual" versus "physical" address stuff in ELF is clearly misspecified in the extreme), but they are clearly not sufficient for presenting a well-defined entry condition. So it's not really anywhere near as straightforward as you make it sound. -hpa - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
--- "Eric W. Biederman" <[EMAIL PROTECTED]> wrote: > What I was thinking is that in the not we place the physical address > and length that we load the real mode code at. My assumption being > that we have marked the real mode code __init or the equivalent, > so we always load and just ignore it later on. But Gujin need the real-mode part linked at zero and there isn't anything linked at this address in the current Linux kernel, no section at all. If you ask to do a separate link then I loose all the symbols and the ELF standard tools are a lot less useable. > Playing the games with the addresses does allow the existing debugging > tools to work without problem because the end users of the code all > examine the virtual not the physical address. The linker also resolve relocation in virtual addresses so this part needs virtual address = 0. > I agree I want a reasonable bootloader as well. > [snip] > ELF should make it much easier for people implementing simple > stand-alone executables for testing. As you don't even need a linker > script or any fancy games if you don't take arguments. Similarly > the switch from linux to multi-boot that almost standard is just > supporting a different argument passing format. There is plenty of possible future, but right now I have a simple (OK, not perfect, but code is shown) solution which works. I can modify bits, but there is no point complexifying the system for possible theoretical problems - by experience you always miss important future problem while overdesigning for some problem which never appear. If someone can show me a real problem - then I'd like to hear from him, but ELF compatibility will not be up to the point where the user will be able to run the kernel from a Xterm command line. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain <[EMAIL PROTECTED]> writes: > --- "Eric W. Biederman" wrote: >> So I really don't care if we move whole real mode section into a note >> or if we just put a pointer to it into a note. What is important is >> that we use a note to find it. > > Well, we can advertise by a note the section number or the section > name which contains the real-mode code, but finding the section of > type SHT_PROGBITS having SHF_EXECINSTR flags linked at zero is not > that difficult to do: it is what Gujin does right now when it does > not find the program header linked at zero. What I was thinking is that in the not we place the physical address and length that we load the real mode code at. My assumption being that we have marked the real mode code __init or the equivalent, so we always load and just ignore it later on. >> Which means that we could do something goofy in the linker script >> like we do with the current vdso. So we could give it a virtual >> address of 0 and a physical address in the init code section. > > Gujin loads at the physical address, i.e. kernel is loaded at > 0x10 and not 0xC010, is that wrong? No. For practical purposes we can say virtual addresses are for the code and the kernel. Physical addresses are for the loader. > I am not sure playing these games with addresses is cleaner than > not loading a section which is not in the program header. It is some cleaner. I won't call it perfectly clean, but it is better. It keeps everything but argument generation generic, and there is already precedent in things like the vdso. Playing the games with the addresses does allow the existing debugging tools to work without problem because the end users of the code all examine the virtual not the physical address. >> For me the objective is not so much reusing the existing tools >> (although that is a plus) but more to be able to build a unified >> binary that can be used for everything, and will give us the freedom >> to do interesting things with the kernel in the future, and hopefully >> something that is more or less usable by portable bootloaders. Having >> a different file format and different rules for different >> architectures is a pain. > > For me the objective is to have a reasonable bootloader, > I will not have the time to port back and test with every other > bootloader some transfert of code from Gujin to an ELF looking > like bzImage file. I agree I want a reasonable bootloader as well. With ELF we get a single file format that works on multiple architectures and for multiple OS-s, with well understood rules. This allows for a broader audience who can review and maintain the code. In addition to the real possibility of multi-architecture or multi-os bootloaders. What ELF cannot standardize is the how we pass the initial arguments. ELF does standardize the initial parameters for unix executables but the requirements of arguments to kernels, and the years of practice are quite different. ELF should make it much easier for people implementing simple stand-alone executables for testing. As you don't even need a linker script or any fancy games if you don't take arguments. Similarly the switch from linux to multi-boot that almost standard is just supporting a different argument passing format. Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
--- "Eric W. Biederman" wrote: > So I really don't care if we move whole real mode section into a note > or if we just put a pointer to it into a note. What is important is > that we use a note to find it. Well, we can advertise by a note the section number or the section name which contains the real-mode code, but finding the section of type SHT_PROGBITS having SHF_EXECINSTR flags linked at zero is not that difficult to do: it is what Gujin does right now when it does not find the program header linked at zero. > Which means that we could do something goofy in the linker script > like we do with the current vdso. So we could give it a virtual > address of 0 and a physical address in the init code section. Gujin loads at the physical address, i.e. kernel is loaded at 0x10 and not 0xC010, is that wrong? I am not sure playing these games with addresses is cleaner than not loading a section which is not in the program header. > For me the objective is not so much reusing the existing tools > (although that is a plus) but more to be able to build a unified > binary that can be used for everything, and will give us the freedom > to do interesting things with the kernel in the future, and hopefully > something that is more or less usable by portable bootloaders. Having > a different file format and different rules for different > architectures is a pain. For me the objective is to have a reasonable bootloader, I will not have the time to port back and test with every other bootloader some transfert of code from Gujin to an ELF looking like bzImage file. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
--- Eric W. Biederman wrote: So I really don't care if we move whole real mode section into a note or if we just put a pointer to it into a note. What is important is that we use a note to find it. Well, we can advertise by a note the section number or the section name which contains the real-mode code, but finding the section of type SHT_PROGBITS having SHF_EXECINSTR flags linked at zero is not that difficult to do: it is what Gujin does right now when it does not find the program header linked at zero. Which means that we could do something goofy in the linker script like we do with the current vdso. So we could give it a virtual address of 0 and a physical address in the init code section. Gujin loads at the physical address, i.e. kernel is loaded at 0x10 and not 0xC010, is that wrong? I am not sure playing these games with addresses is cleaner than not loading a section which is not in the program header. For me the objective is not so much reusing the existing tools (although that is a plus) but more to be able to build a unified binary that can be used for everything, and will give us the freedom to do interesting things with the kernel in the future, and hopefully something that is more or less usable by portable bootloaders. Having a different file format and different rules for different architectures is a pain. For me the objective is to have a reasonable bootloader, I will not have the time to port back and test with every other bootloader some transfert of code from Gujin to an ELF looking like bzImage file. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain [EMAIL PROTECTED] writes: --- Eric W. Biederman wrote: So I really don't care if we move whole real mode section into a note or if we just put a pointer to it into a note. What is important is that we use a note to find it. Well, we can advertise by a note the section number or the section name which contains the real-mode code, but finding the section of type SHT_PROGBITS having SHF_EXECINSTR flags linked at zero is not that difficult to do: it is what Gujin does right now when it does not find the program header linked at zero. What I was thinking is that in the not we place the physical address and length that we load the real mode code at. My assumption being that we have marked the real mode code __init or the equivalent, so we always load and just ignore it later on. Which means that we could do something goofy in the linker script like we do with the current vdso. So we could give it a virtual address of 0 and a physical address in the init code section. Gujin loads at the physical address, i.e. kernel is loaded at 0x10 and not 0xC010, is that wrong? No. For practical purposes we can say virtual addresses are for the code and the kernel. Physical addresses are for the loader. I am not sure playing these games with addresses is cleaner than not loading a section which is not in the program header. It is some cleaner. I won't call it perfectly clean, but it is better. It keeps everything but argument generation generic, and there is already precedent in things like the vdso. Playing the games with the addresses does allow the existing debugging tools to work without problem because the end users of the code all examine the virtual not the physical address. For me the objective is not so much reusing the existing tools (although that is a plus) but more to be able to build a unified binary that can be used for everything, and will give us the freedom to do interesting things with the kernel in the future, and hopefully something that is more or less usable by portable bootloaders. Having a different file format and different rules for different architectures is a pain. For me the objective is to have a reasonable bootloader, I will not have the time to port back and test with every other bootloader some transfert of code from Gujin to an ELF looking like bzImage file. I agree I want a reasonable bootloader as well. With ELF we get a single file format that works on multiple architectures and for multiple OS-s, with well understood rules. This allows for a broader audience who can review and maintain the code. In addition to the real possibility of multi-architecture or multi-os bootloaders. What ELF cannot standardize is the how we pass the initial arguments. ELF does standardize the initial parameters for unix executables but the requirements of arguments to kernels, and the years of practice are quite different. ELF should make it much easier for people implementing simple stand-alone executables for testing. As you don't even need a linker script or any fancy games if you don't take arguments. Similarly the switch from linux to multi-boot that almost standard is just supporting a different argument passing format. Eric - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
--- Eric W. Biederman [EMAIL PROTECTED] wrote: What I was thinking is that in the not we place the physical address and length that we load the real mode code at. My assumption being that we have marked the real mode code __init or the equivalent, so we always load and just ignore it later on. But Gujin need the real-mode part linked at zero and there isn't anything linked at this address in the current Linux kernel, no section at all. If you ask to do a separate link then I loose all the symbols and the ELF standard tools are a lot less useable. Playing the games with the addresses does allow the existing debugging tools to work without problem because the end users of the code all examine the virtual not the physical address. The linker also resolve relocation in virtual addresses so this part needs virtual address = 0. I agree I want a reasonable bootloader as well. [snip] ELF should make it much easier for people implementing simple stand-alone executables for testing. As you don't even need a linker script or any fancy games if you don't take arguments. Similarly the switch from linux to multi-boot that almost standard is just supporting a different argument passing format. There is plenty of possible future, but right now I have a simple (OK, not perfect, but code is shown) solution which works. I can modify bits, but there is no point complexifying the system for possible theoretical problems - by experience you always miss important future problem while overdesigning for some problem which never appear. If someone can show me a real problem - then I'd like to hear from him, but ELF compatibility will not be up to the point where the user will be able to run the kernel from a Xterm command line. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Eric W. Biederman wrote: With ELF we get a single file format that works on multiple architectures and for multiple OS-s, with well understood rules. This allows for a broader audience who can review and maintain the code. In addition to the real possibility of multi-architecture or multi-os bootloaders. Well understood MAYBE (although the virtual versus physical address stuff in ELF is clearly misspecified in the extreme), but they are clearly not sufficient for presenting a well-defined entry condition. So it's not really anywhere near as straightforward as you make it sound. -hpa - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
H. Peter Anvin [EMAIL PROTECTED] writes: Eric W. Biederman wrote: With ELF we get a single file format that works on multiple architectures and for multiple OS-s, with well understood rules. This allows for a broader audience who can review and maintain the code. In addition to the real possibility of multi-architecture or multi-os bootloaders. Well understood MAYBE (although the virtual versus physical address stuff in ELF is clearly misspecified in the extreme), but they are clearly not sufficient for presenting a well-defined entry condition. So it's not really anywhere near as straightforward as you make it sound. About the entry condition I completely agree. The fact that the generic tools almost do what you need and then fall short is also problematic. However it still appears a reasonable approach to consider. Eric - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: RE : Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain <[EMAIL PROTECTED]> writes: > Eric W. Biederman wrote: >> Etienne Lorrain writes: >> > Well, a self relocating image cannot be an ELF file because the code >> > to relocate the ELF cannot be executed at the wrong place. >> > If relocation is needed, I would better like not to link vmlinux at a >> > fixed address first. In fact I wonder if we are talking of the same >> > kind of relocation: you seem to talk about "ld --pic-executable" while >> > I am thinking of "ld -r" to "locate" it at the bootloader loading time. >> > The main problem I see is that I do not have the code for that, and >> > I am going deeper/earlier into the generation of vmlinux, while comments >> > are "already you are too early, loading an ELF file is too complex for >> > a bootloader". The solution I have already is working. >> >> Being very clear. ld --pic-executable or ld -shared is essentially >> what we are talking when we are discussing building a relocatable >> kernel. Something with the properties of an ELF ET_DYN executable >> that does not use an interpreter. ld.so is the only common executable >> of this type in linux. > > So we are talking of two different things. > You want to _compile_ the totality of the kernel as code and data > relocatable - right now only modules are compiled that way. > On IA32 architecture, there is no problem having the code relocatable, > because most/all calls and jumps are PC relative, so very few instructions > need to be different. > The problem is having the data relocatable, then every access to memory > has to be relative to a register, and so the register EBX is reserved > for that by the compiler. > The IA32 processor does not have enough registers already, removing one > of them makes the assembly seriously worse - mostly when dealing with > 64 bits "long long" - you can only get two of them in register and a > simple addition produces long and slow code. > Having the data relocatable could be done by using the base address of > IA32 segments, but this does not seem to be the preferred method because > segments use seems to slow down the processor a lot more. There are two more options in addition to segments: - Running at a fixed virtual address. - Processing relocation entries to modify the initial code/data to live where we want. We don't have the that shared libraries do of wanting to share text/data pages loaded at different addresses. > What I was talking of was "loading" a non relocatable kernel (compiled > as usual without -fPIC/-shared) at any address, so not inducing slow > down whatever the base address used. I said that it is possible, maybe > quite easy to do so, but I do not have the code for it - and people > say right now that an ELF bootloader is not what they want to. The thing to pick up on is that we really don't want the gcc flag -fPIC but if ld wasn't an idiot we would want the linker flag -shared. Anyway we have working code merged that processes all of the relocation entries and modifies the kernel to run properly where we load it at. This has all been done and merged at this point. It's only available to a bzImage loader but it is all there. Currently the only overhead is something like 5 or 10 percent overhead to hold the relocation entries. The only thing currently compiled with -fPIC is the kernel gzip decompressor and it is carefully coded so it doesn't generate any relocation entries that need processing. It is just throw away code so we don't much care about it's efficiency. > Now, if people wants to switch to a completely relocatable kernel, > because they do not mind the slow down (that I am not really able > to quantify) - and because anyway most of their kernel is modules, > then Gujin can relocate that without _any_ problem; in fact it > does already load in intermediate (HIMEM malloc'ed) memory under > DOS to relocate it later (called 'late relocation' in Gujin code, > done every times the code is not at the right address in memory, > interrupt disabled and in protected mode). Yes. That is roughly what we are saying. We want that at least as a well defined option. > Loading and executing at whatever address is not a problem, > and should be doable already in the real-mode function of the > kernel called by Gujin, by modifying few elements of the > structure passed in parameter of this real-mode function: > it should be done after loading the E820 memory map for the > BIOS parameter page. > I am reading right now Gujin source and note that you may want > to access array elf_reloc[] and variable nb_elf_reloc in this > Linux function - will be added in next Gujin version. > You already can access type LOADER_str parameter "loader" > where you can get runadr which can overwrite the ELF entry > point address. The old ELF header is at loader.load_address > if it is needed. > > The main problem Gujin has, is to decide where to load a > relocatable kernel, so this policy is left to the kernel and
RE : Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Eric W. Biederman wrote: > Etienne Lorrain writes: > > Well, a self relocating image cannot be an ELF file because the code > > to relocate the ELF cannot be executed at the wrong place. > > If relocation is needed, I would better like not to link vmlinux at a > > fixed address first. In fact I wonder if we are talking of the same > > kind of relocation: you seem to talk about "ld --pic-executable" while > > I am thinking of "ld -r" to "locate" it at the bootloader loading time. > > The main problem I see is that I do not have the code for that, and > > I am going deeper/earlier into the generation of vmlinux, while comments > > are "already you are too early, loading an ELF file is too complex for > > a bootloader". The solution I have already is working. > > Being very clear. ld --pic-executable or ld -shared is essentially > what we are talking when we are discussing building a relocatable > kernel. Something with the properties of an ELF ET_DYN executable > that does not use an interpreter. ld.so is the only common executable > of this type in linux. So we are talking of two different things. You want to _compile_ the totality of the kernel as code and data relocatable - right now only modules are compiled that way. On IA32 architecture, there is no problem having the code relocatable, because most/all calls and jumps are PC relative, so very few instructions need to be different. The problem is having the data relocatable, then every access to memory has to be relative to a register, and so the register EBX is reserved for that by the compiler. The IA32 processor does not have enough registers already, removing one of them makes the assembly seriously worse - mostly when dealing with 64 bits "long long" - you can only get two of them in register and a simple addition produces long and slow code. Having the data relocatable could be done by using the base address of IA32 segments, but this does not seem to be the preferred method because segments use seems to slow down the processor a lot more. What I was talking of was "loading" a non relocatable kernel (compiled as usual without -fPIC/-shared) at any address, so not inducing slow down whatever the base address used. I said that it is possible, maybe quite easy to do so, but I do not have the code for it - and people say right now that an ELF bootloader is not what they want to. Now, if people wants to switch to a completely relocatable kernel, because they do not mind the slow down (that I am not really able to quantify) - and because anyway most of their kernel is modules, then Gujin can relocate that without _any_ problem; in fact it does already load in intermediate (HIMEM malloc'ed) memory under DOS to relocate it later (called 'late relocation' in Gujin code, done every times the code is not at the right address in memory, interrupt disabled and in protected mode). Loading and executing at whatever address is not a problem, and should be doable already in the real-mode function of the kernel called by Gujin, by modifying few elements of the structure passed in parameter of this real-mode function: it should be done after loading the E820 memory map for the BIOS parameter page. I am reading right now Gujin source and note that you may want to access array elf_reloc[] and variable nb_elf_reloc in this Linux function - will be added in next Gujin version. You already can access type LOADER_str parameter "loader" where you can get runadr which can overwrite the ELF entry point address. The old ELF header is at loader.load_address if it is needed. The main problem Gujin has, is to decide where to load a relocatable kernel, so this policy is left to the kernel and not to the user. For instance and as you made me think of, if you do not want for security reasons that the kernel memory could be modified by an attacker by using the legacy DMA, then you want to load it at an address which cannot be modified this way, i.e. over 16 Mbytes. Note that some chipset had an extension I/O address to extend this legacy DMA to 32 bits addresses, but it has not been documented and probably not implemented widely. I do not know how many people are like me, but I am currently typing on a monolitic kernel because I find it a _lot_ simpler to do a ".config" once and make it slowly evolve with new kernel versions (answering No to most of the questions), than managing hundreds of module files each time I regenerate/remove a kernel. The kernel I am working on is not relocatable because - well I do not have crash to debug... So I think that from the Gujin point of view, everything you need is done and working (but the access to elf_reloc)- please explain what you want more if not. If you want a solution to load now, you can use patch linux-2.6.20-rc5 in sourceforge, which uses an intermediate binary dump of the ELF file, and can be relocated the non ELF way. > > In fact, thinking more about that, I am going back to my implementation >
RE : Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Eric W. Biederman wrote: Etienne Lorrain writes: Well, a self relocating image cannot be an ELF file because the code to relocate the ELF cannot be executed at the wrong place. If relocation is needed, I would better like not to link vmlinux at a fixed address first. In fact I wonder if we are talking of the same kind of relocation: you seem to talk about ld --pic-executable while I am thinking of ld -r to locate it at the bootloader loading time. The main problem I see is that I do not have the code for that, and I am going deeper/earlier into the generation of vmlinux, while comments are already you are too early, loading an ELF file is too complex for a bootloader. The solution I have already is working. Being very clear. ld --pic-executable or ld -shared is essentially what we are talking when we are discussing building a relocatable kernel. Something with the properties of an ELF ET_DYN executable that does not use an interpreter. ld.so is the only common executable of this type in linux. So we are talking of two different things. You want to _compile_ the totality of the kernel as code and data relocatable - right now only modules are compiled that way. On IA32 architecture, there is no problem having the code relocatable, because most/all calls and jumps are PC relative, so very few instructions need to be different. The problem is having the data relocatable, then every access to memory has to be relative to a register, and so the register EBX is reserved for that by the compiler. The IA32 processor does not have enough registers already, removing one of them makes the assembly seriously worse - mostly when dealing with 64 bits long long - you can only get two of them in register and a simple addition produces long and slow code. Having the data relocatable could be done by using the base address of IA32 segments, but this does not seem to be the preferred method because segments use seems to slow down the processor a lot more. What I was talking of was loading a non relocatable kernel (compiled as usual without -fPIC/-shared) at any address, so not inducing slow down whatever the base address used. I said that it is possible, maybe quite easy to do so, but I do not have the code for it - and people say right now that an ELF bootloader is not what they want to. Now, if people wants to switch to a completely relocatable kernel, because they do not mind the slow down (that I am not really able to quantify) - and because anyway most of their kernel is modules, then Gujin can relocate that without _any_ problem; in fact it does already load in intermediate (HIMEM malloc'ed) memory under DOS to relocate it later (called 'late relocation' in Gujin code, done every times the code is not at the right address in memory, interrupt disabled and in protected mode). Loading and executing at whatever address is not a problem, and should be doable already in the real-mode function of the kernel called by Gujin, by modifying few elements of the structure passed in parameter of this real-mode function: it should be done after loading the E820 memory map for the BIOS parameter page. I am reading right now Gujin source and note that you may want to access array elf_reloc[] and variable nb_elf_reloc in this Linux function - will be added in next Gujin version. You already can access type LOADER_str parameter loader where you can get runadr which can overwrite the ELF entry point address. The old ELF header is at loader.load_address if it is needed. The main problem Gujin has, is to decide where to load a relocatable kernel, so this policy is left to the kernel and not to the user. For instance and as you made me think of, if you do not want for security reasons that the kernel memory could be modified by an attacker by using the legacy DMA, then you want to load it at an address which cannot be modified this way, i.e. over 16 Mbytes. Note that some chipset had an extension I/O address to extend this legacy DMA to 32 bits addresses, but it has not been documented and probably not implemented widely. I do not know how many people are like me, but I am currently typing on a monolitic kernel because I find it a _lot_ simpler to do a .config once and make it slowly evolve with new kernel versions (answering No to most of the questions), than managing hundreds of module files each time I regenerate/remove a kernel. The kernel I am working on is not relocatable because - well I do not have crash to debug... So I think that from the Gujin point of view, everything you need is done and working (but the access to elf_reloc)- please explain what you want more if not. If you want a solution to load now, you can use patch linux-2.6.20-rc5 in sourceforge, which uses an intermediate binary dump of the ELF file, and can be relocated the non ELF way. In fact, thinking more about that, I am going back to my implementation of it, because on ia32 the interrupt vectors
Re: RE : Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain [EMAIL PROTECTED] writes: Eric W. Biederman wrote: Etienne Lorrain writes: Well, a self relocating image cannot be an ELF file because the code to relocate the ELF cannot be executed at the wrong place. If relocation is needed, I would better like not to link vmlinux at a fixed address first. In fact I wonder if we are talking of the same kind of relocation: you seem to talk about ld --pic-executable while I am thinking of ld -r to locate it at the bootloader loading time. The main problem I see is that I do not have the code for that, and I am going deeper/earlier into the generation of vmlinux, while comments are already you are too early, loading an ELF file is too complex for a bootloader. The solution I have already is working. Being very clear. ld --pic-executable or ld -shared is essentially what we are talking when we are discussing building a relocatable kernel. Something with the properties of an ELF ET_DYN executable that does not use an interpreter. ld.so is the only common executable of this type in linux. So we are talking of two different things. You want to _compile_ the totality of the kernel as code and data relocatable - right now only modules are compiled that way. On IA32 architecture, there is no problem having the code relocatable, because most/all calls and jumps are PC relative, so very few instructions need to be different. The problem is having the data relocatable, then every access to memory has to be relative to a register, and so the register EBX is reserved for that by the compiler. The IA32 processor does not have enough registers already, removing one of them makes the assembly seriously worse - mostly when dealing with 64 bits long long - you can only get two of them in register and a simple addition produces long and slow code. Having the data relocatable could be done by using the base address of IA32 segments, but this does not seem to be the preferred method because segments use seems to slow down the processor a lot more. There are two more options in addition to segments: - Running at a fixed virtual address. - Processing relocation entries to modify the initial code/data to live where we want. We don't have the that shared libraries do of wanting to share text/data pages loaded at different addresses. What I was talking of was loading a non relocatable kernel (compiled as usual without -fPIC/-shared) at any address, so not inducing slow down whatever the base address used. I said that it is possible, maybe quite easy to do so, but I do not have the code for it - and people say right now that an ELF bootloader is not what they want to. The thing to pick up on is that we really don't want the gcc flag -fPIC but if ld wasn't an idiot we would want the linker flag -shared. Anyway we have working code merged that processes all of the relocation entries and modifies the kernel to run properly where we load it at. This has all been done and merged at this point. It's only available to a bzImage loader but it is all there. Currently the only overhead is something like 5 or 10 percent overhead to hold the relocation entries. The only thing currently compiled with -fPIC is the kernel gzip decompressor and it is carefully coded so it doesn't generate any relocation entries that need processing. It is just throw away code so we don't much care about it's efficiency. Now, if people wants to switch to a completely relocatable kernel, because they do not mind the slow down (that I am not really able to quantify) - and because anyway most of their kernel is modules, then Gujin can relocate that without _any_ problem; in fact it does already load in intermediate (HIMEM malloc'ed) memory under DOS to relocate it later (called 'late relocation' in Gujin code, done every times the code is not at the right address in memory, interrupt disabled and in protected mode). Yes. That is roughly what we are saying. We want that at least as a well defined option. Loading and executing at whatever address is not a problem, and should be doable already in the real-mode function of the kernel called by Gujin, by modifying few elements of the structure passed in parameter of this real-mode function: it should be done after loading the E820 memory map for the BIOS parameter page. I am reading right now Gujin source and note that you may want to access array elf_reloc[] and variable nb_elf_reloc in this Linux function - will be added in next Gujin version. You already can access type LOADER_str parameter loader where you can get runadr which can overwrite the ELF entry point address. The old ELF header is at loader.load_address if it is needed. The main problem Gujin has, is to decide where to load a relocatable kernel, so this policy is left to the kernel and not to the user. For instance and as you made me think of, if you do not want for security reasons that
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain <[EMAIL PROTECTED]> writes: > Well, a self relocating image cannot be an ELF file because the code > to relocate the ELF cannot be executed at the wrong place. > If relocation is needed, I would better like not to link vmlinux at a > fixed address first. In fact I wonder if we are talking of the same > kind of relocation: you seem to talk about "ld --pic-executable" while > I am thinking of "ld -r" to "locate" it at the bootloader loading time. > The main problem I see is that I do not have the code for that, and > I am going deeper/earlier into the generation of vmlinux, while comments > are "already you are too early, loading an ELF file is too complex for > a bootloader". The solution I have already is working. Being very clear. ld --pic-executable or ld -shared is essentially what we are talking when we are discussing building a relocatable kernel. Something with the properties of an ELF ET_DYN executable that does not use an interpreter. ld.so is the only common executable of this type in linux. Loading an ELF executable is very much: - Walk through the program headers and for each PT_LOAD segment load it at the address it requests. - Jump to e_entry from the ELF header. If you are working with a relocatable ELF object the rules become: - Walk through the program header once to find the size and alignment of the chunk of memory that the linux kernel needs. - Find a hole in the memory map that meets those requirements. - Compute the offset - Walk through the program headers and for each PT_LOAD segment add offset to the addresses and load the segment like normal. - Jump to offset + e_entry. This is within the scope of what a bootloader can reasonably do, and I have implemented it in etherboot as well as /sbin/kexec. >> > If you cannot get a PT_LOAD >> > section, maybe we can put a simple system in NOTE, or just create a >> > PT_LOAD16 if the linker accepts other values. >> >> My guess is that PT_LOAD16 is not an acceptable value. Putting information >> in PT_NOTE seems interesting (As Eric already mentioned). > > In fact, thinking more about that, I am going back to my implementation > of it, because on ia32 the interrupt vectors are at address zero and it is > obviouly an invalid address to load an ELF for this architecture. No special games no special rules with the well defined ELF components either add a note that you can define all of the semantics yourself or don't do it. That is what the notes are there for. > But for the linker, it is the right address to link it (being an offset > into a non-null segment in real mode), and because the entry point has > to be zero (I cannot use the ELF entry value) the program header base > address has to be zero. Agreed. When the object file is linked using offset 0, and letting the real mode segments do have different bases to do your relocation is fine. > Anyway, your loader in (probably) written in C, so a test against zero > is a simple thing to do, and should be done anyway to check for an > incorrect ELF program header. I wonder if this NOTE program header is > not simply designed as an "end" marker, it does not seem to contain > anything, so me defining the realmode after that program header may > be a good idea. We have been very sparse on the usage of ELF notes but yes they exist and yes people do look at them. Please dig up a copy of the ELF spec and read up on them or look at etherboot for an example. > If you really are tring to catch an erroneous DMA into the kernel, > is it better to keep an exact copy of the kernel you are using somewhere > else to do a bit-to-bit comparisson after the crash, and so no relocate. > Anyway if the DMA crash has crashed the exception handling area the system > is dead anyway. No. We are not trying to catch a erroneous DMA in that sense. We do not shutdown any drivers when switching to the new kernel from panic(), because we don't know what is broken. Any single bit of kernel code of could cause problems not just the exception table. Running in an area that we have never used for DMA and is completely reserved gives us freedom to not worry about it. I.e. This is not error detection but future error prevention and it works. Before starting the new kernel we do a sha256 checksum test on the new kernel and on our code that is running the checksum. All of which comes from /sbin/kexec. Not compiled into the running kernel. The policy is in user space. >> Interesting question, How does a boot loader/user decide where to load >> the relocatable image? I think it depends on the new interesting usages >> of the relocatable kernel. As of today, kexec knows where is reserved >> memory region (Read from /proc/iomem) and it loads the image at the >> start of that reserved region (Meeting alignment restrictions, if any). So >> in this case boot loader takes the decision. May be a user option also >> can be created, something like
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
--- Vivek Goyal <[EMAIL PROTECTED]> wrote: > No, as of today, kexec does not relocate vmlinux. At compilation time, > vmlinux is compiled for a fixed address and vmlinux is loaded at that > address. This compile address can be controlled with CONFIG_PHYSICAL_START, > as you already mentioned in your patchset. > > So in the case of kdump, if one wishes to use vmlinux instead of a relocatable > bzImage, he will re-compile the kernel for a different address and then use > that vmlinux. > > OTOH, boot loader can possibly relocate vmlinux because all the relocation > information is retained in vmlinux (Using ld option --emit-relocs). But > probably self relocating image should be a better option as all the > boot-loaders out there don't have to be modifed. Well, a self relocating image cannot be an ELF file because the code to relocate the ELF cannot be executed at the wrong place. If relocation is needed, I would better like not to link vmlinux at a fixed address first. In fact I wonder if we are talking of the same kind of relocation: you seem to talk about "ld --pic-executable" while I am thinking of "ld -r" to "locate" it at the bootloader loading time. The main problem I see is that I do not have the code for that, and I am going deeper/earlier into the generation of vmlinux, while comments are "already you are too early, loading an ELF file is too complex for a bootloader". The solution I have already is working. > > If you cannot get a PT_LOAD > > section, maybe we can put a simple system in NOTE, or just create a > > PT_LOAD16 if the linker accepts other values. > > My guess is that PT_LOAD16 is not an acceptable value. Putting information > in PT_NOTE seems interesting (As Eric already mentioned). In fact, thinking more about that, I am going back to my implementation of it, because on ia32 the interrupt vectors are at address zero and it is obviouly an invalid address to load an ELF for this architecture. But for the linker, it is the right address to link it (being an offset into a non-null segment in real mode), and because the entry point has to be zero (I cannot use the ELF entry value) the program header base address has to be zero. Anyway, your loader in (probably) written in C, so a test against zero is a simple thing to do, and should be done anyway to check for an incorrect ELF program header. I wonder if this NOTE program header is not simply designed as an "end" marker, it does not seem to contain anything, so me defining the realmode after that program header may be a good idea. > We already do checksum verification to make sure newly loaded kernel has > not been corrupted before we jump to it. What's the point in doing the > verification existing kernel text. Because even that is intact, and you > reload your data section, that data section will have to be reloaded at > the same place where previous data section was. Now after you re-loaded > the data section, some DMA might corrupt it now (Because we never stopped > DMAs). So loading a fresh copy of text and data section at a reserved > memory location seems safer. If you really are tring to catch an erroneous DMA into the kernel, is it better to keep an exact copy of the kernel you are using somewhere else to do a bit-to-bit comparisson after the crash, and so no relocate. Anyway if the DMA crash has crashed the exception handling area the system is dead anyway. > Interesting question, How does a boot loader/user decide where to load > the relocatable image? I think it depends on the new interesting usages > of the relocatable kernel. As of today, kexec knows where is reserved > memory region (Read from /proc/iomem) and it loads the image at the > start of that reserved region (Meeting alignment restrictions, if any). So > in this case boot loader takes the decision. May be a user option also > can be created, something like --load-address=0xXYZ and then people > can have fun loading same image at various addresses. I think that you are asking too much for the bootloader user, and that is a decision he has to take *before* the crash; even me, I would select one address like 16 Mbytes and stick with it... If the running Linux kernel do not erase Gujin from memory, it could also go back to real mode and do a "longjmp()" to return to the Gujin interface - but most of the times the system had a reason to crash (for instance a ventilator stopped working) and you can plan whatever you want in software... Thanks, Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
--- Vivek Goyal [EMAIL PROTECTED] wrote: No, as of today, kexec does not relocate vmlinux. At compilation time, vmlinux is compiled for a fixed address and vmlinux is loaded at that address. This compile address can be controlled with CONFIG_PHYSICAL_START, as you already mentioned in your patchset. So in the case of kdump, if one wishes to use vmlinux instead of a relocatable bzImage, he will re-compile the kernel for a different address and then use that vmlinux. OTOH, boot loader can possibly relocate vmlinux because all the relocation information is retained in vmlinux (Using ld option --emit-relocs). But probably self relocating image should be a better option as all the boot-loaders out there don't have to be modifed. Well, a self relocating image cannot be an ELF file because the code to relocate the ELF cannot be executed at the wrong place. If relocation is needed, I would better like not to link vmlinux at a fixed address first. In fact I wonder if we are talking of the same kind of relocation: you seem to talk about ld --pic-executable while I am thinking of ld -r to locate it at the bootloader loading time. The main problem I see is that I do not have the code for that, and I am going deeper/earlier into the generation of vmlinux, while comments are already you are too early, loading an ELF file is too complex for a bootloader. The solution I have already is working. If you cannot get a PT_LOAD section, maybe we can put a simple system in NOTE, or just create a PT_LOAD16 if the linker accepts other values. My guess is that PT_LOAD16 is not an acceptable value. Putting information in PT_NOTE seems interesting (As Eric already mentioned). In fact, thinking more about that, I am going back to my implementation of it, because on ia32 the interrupt vectors are at address zero and it is obviouly an invalid address to load an ELF for this architecture. But for the linker, it is the right address to link it (being an offset into a non-null segment in real mode), and because the entry point has to be zero (I cannot use the ELF entry value) the program header base address has to be zero. Anyway, your loader in (probably) written in C, so a test against zero is a simple thing to do, and should be done anyway to check for an incorrect ELF program header. I wonder if this NOTE program header is not simply designed as an end marker, it does not seem to contain anything, so me defining the realmode after that program header may be a good idea. We already do checksum verification to make sure newly loaded kernel has not been corrupted before we jump to it. What's the point in doing the verification existing kernel text. Because even that is intact, and you reload your data section, that data section will have to be reloaded at the same place where previous data section was. Now after you re-loaded the data section, some DMA might corrupt it now (Because we never stopped DMAs). So loading a fresh copy of text and data section at a reserved memory location seems safer. If you really are tring to catch an erroneous DMA into the kernel, is it better to keep an exact copy of the kernel you are using somewhere else to do a bit-to-bit comparisson after the crash, and so no relocate. Anyway if the DMA crash has crashed the exception handling area the system is dead anyway. Interesting question, How does a boot loader/user decide where to load the relocatable image? I think it depends on the new interesting usages of the relocatable kernel. As of today, kexec knows where is reserved memory region (Read from /proc/iomem) and it loads the image at the start of that reserved region (Meeting alignment restrictions, if any). So in this case boot loader takes the decision. May be a user option also can be created, something like --load-address=0xXYZ and then people can have fun loading same image at various addresses. I think that you are asking too much for the bootloader user, and that is a decision he has to take *before* the crash; even me, I would select one address like 16 Mbytes and stick with it... If the running Linux kernel do not erase Gujin from memory, it could also go back to real mode and do a longjmp() to return to the Gujin interface - but most of the times the system had a reason to crash (for instance a ventilator stopped working) and you can plan whatever you want in software... Thanks, Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain [EMAIL PROTECTED] writes: Well, a self relocating image cannot be an ELF file because the code to relocate the ELF cannot be executed at the wrong place. If relocation is needed, I would better like not to link vmlinux at a fixed address first. In fact I wonder if we are talking of the same kind of relocation: you seem to talk about ld --pic-executable while I am thinking of ld -r to locate it at the bootloader loading time. The main problem I see is that I do not have the code for that, and I am going deeper/earlier into the generation of vmlinux, while comments are already you are too early, loading an ELF file is too complex for a bootloader. The solution I have already is working. Being very clear. ld --pic-executable or ld -shared is essentially what we are talking when we are discussing building a relocatable kernel. Something with the properties of an ELF ET_DYN executable that does not use an interpreter. ld.so is the only common executable of this type in linux. Loading an ELF executable is very much: - Walk through the program headers and for each PT_LOAD segment load it at the address it requests. - Jump to e_entry from the ELF header. If you are working with a relocatable ELF object the rules become: - Walk through the program header once to find the size and alignment of the chunk of memory that the linux kernel needs. - Find a hole in the memory map that meets those requirements. - Compute the offset - Walk through the program headers and for each PT_LOAD segment add offset to the addresses and load the segment like normal. - Jump to offset + e_entry. This is within the scope of what a bootloader can reasonably do, and I have implemented it in etherboot as well as /sbin/kexec. If you cannot get a PT_LOAD section, maybe we can put a simple system in NOTE, or just create a PT_LOAD16 if the linker accepts other values. My guess is that PT_LOAD16 is not an acceptable value. Putting information in PT_NOTE seems interesting (As Eric already mentioned). In fact, thinking more about that, I am going back to my implementation of it, because on ia32 the interrupt vectors are at address zero and it is obviouly an invalid address to load an ELF for this architecture. No special games no special rules with the well defined ELF components either add a note that you can define all of the semantics yourself or don't do it. That is what the notes are there for. But for the linker, it is the right address to link it (being an offset into a non-null segment in real mode), and because the entry point has to be zero (I cannot use the ELF entry value) the program header base address has to be zero. Agreed. When the object file is linked using offset 0, and letting the real mode segments do have different bases to do your relocation is fine. Anyway, your loader in (probably) written in C, so a test against zero is a simple thing to do, and should be done anyway to check for an incorrect ELF program header. I wonder if this NOTE program header is not simply designed as an end marker, it does not seem to contain anything, so me defining the realmode after that program header may be a good idea. We have been very sparse on the usage of ELF notes but yes they exist and yes people do look at them. Please dig up a copy of the ELF spec and read up on them or look at etherboot for an example. If you really are tring to catch an erroneous DMA into the kernel, is it better to keep an exact copy of the kernel you are using somewhere else to do a bit-to-bit comparisson after the crash, and so no relocate. Anyway if the DMA crash has crashed the exception handling area the system is dead anyway. No. We are not trying to catch a erroneous DMA in that sense. We do not shutdown any drivers when switching to the new kernel from panic(), because we don't know what is broken. Any single bit of kernel code of could cause problems not just the exception table. Running in an area that we have never used for DMA and is completely reserved gives us freedom to not worry about it. I.e. This is not error detection but future error prevention and it works. Before starting the new kernel we do a sha256 checksum test on the new kernel and on our code that is running the checksum. All of which comes from /sbin/kexec. Not compiled into the running kernel. The policy is in user space. Interesting question, How does a boot loader/user decide where to load the relocatable image? I think it depends on the new interesting usages of the relocatable kernel. As of today, kexec knows where is reserved memory region (Read from /proc/iomem) and it loads the image at the start of that reserved region (Meeting alignment restrictions, if any). So in this case boot loader takes the decision. May be a user option also can be created, something like --load-address=0xXYZ and then people can have fun loading same image at various
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
On Thu, Feb 08, 2007 at 01:10:58PM +, Etienne Lorrain wrote: > > > I assume you care about this ELF header because you are also a user of > > > the ELF file vmlinux, aren't you? > > > > Yes I am. I use kexec boot loader which has capability to load ELF kernel > > images (vmlinux). That's why I am concerned about linking real mode code > > in vmlinux as for kdump case I shall have to be aware that kernel vmlinux > > might contain a special PT_LOAD type program header which will contain > > real mode code and it does not have to be loaded. Then I will run into > > guessing business which one is that real mode PT_LOAD program header and > > my assumption might very well break in next few kernel release. > > So is kexec able to relocate this vmlinux? If kexec and Gujin do > approximately > the same thing they should do it the same way. No, as of today, kexec does not relocate vmlinux. At compilation time, vmlinux is compiled for a fixed address and vmlinux is loaded at that address. This compile address can be controlled with CONFIG_PHYSICAL_START, as you already mentioned in your patchset. So in the case of kdump, if one wishes to use vmlinux instead of a relocatable bzImage, he will re-compile the kernel for a different address and then use that vmlinux. OTOH, boot loader can possibly relocate vmlinux because all the relocation information is retained in vmlinux (Using ld option --emit-relocs). But probably self relocating image should be a better option as all the boot-loaders out there don't have to be modifed. > If you cannot get a PT_LOAD > section, maybe we can put a simple system in NOTE, or just create a > PT_LOAD16 if the linker accepts other values. My guess is that PT_LOAD16 is not an acceptable value. Putting information in PT_NOTE seems interesting (As Eric already mentioned). > I do not really like to relocate after vmlinux has been linked at a fixed > address, > because I am not sure you can guess each address to relocate or not: > you can define permanent address in the linker file by simply "symbol = > address" > and those should never be relocated. For instance the very high addresses > on ia32 may point to registers or FPU instead of memory, so may or may not > have to be relocated. I also would better like you not to relocate the > real-mode > addresses. > I think there will have to be an upper limit on the address where you can relocate your image to. Anyway, given i386 virtual memory address space constraint, I think you can not load kernel image in high memory (896MB). It has to run anywhere below that. I think you are referring to defining absolute symbols. Already there are some absolute symbols generated when kernel is compiled and these are not relocated while image is being loaded at a non-compiled address. So if a boot loader decides to relocate a vmlinux image, it will have to skip relocations present w.r.t absolute symbols. (Unfortunately, ld does generate relocation entries even for absolute symbols). > > > And you do not want to write protect the kernel code (if the CPU write > > > protection > > > is not working, the hardware is not working so debug will be difficult, > > > and a simple > > > CRC32 can tell kernel memory failure) and use twice the same code memory > > > (with different data area or saving kernel data elsewhere before reload). > > > Is that related to module loading or instruction set detection/patch or > > > multiprocessor? > > > > If I understand it right, you seem to be suggesting that I don't have to > > reload the kernel text and I can only reload the data for second kernel? > > > > We run the whole of the kernel from a mutually execlusive location from > > first kernel to mitigate the concerns that first kernel's ongoing DMA might > > corrupt second kernel. That's why first kernel's text can't be reused. > > So in some exception handler, you detect something is wrong and then > jump to a new kernel. In a nutshell, Yes. > Maybe this class of bug (DMA or hardware bit flipped) > should be detected even without this double kernel environment, by running a > CRC32 on the kernel text section in this exception handler, displayed that in > the crash dump. We already do checksum verification to make sure newly loaded kernel has not been corrupted before we jump to it. What's the point in doing the verification existing kernel text. Because even that is intact, and you reload your data section, that data section will have to be reloaded at the same place where previous data section was. Now after you re-loaded the data section, some DMA might corrupt it now (Because we never stopped DMAs). So loading a fresh copy of text and data section at a reserved memory location seems safer. > > > Secondly, it gives flexibility to user that either he can choose to use > > the production kernel as capture kernel or an entirely different custom > > kernel can be used as capture kernel. > > IHMO if it is just capturing the
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
> > I assume you care about this ELF header because you are also a user of > > the ELF file vmlinux, aren't you? > > Yes I am. I use kexec boot loader which has capability to load ELF kernel > images (vmlinux). That's why I am concerned about linking real mode code > in vmlinux as for kdump case I shall have to be aware that kernel vmlinux > might contain a special PT_LOAD type program header which will contain > real mode code and it does not have to be loaded. Then I will run into > guessing business which one is that real mode PT_LOAD program header and > my assumption might very well break in next few kernel release. So is kexec able to relocate this vmlinux? If kexec and Gujin do approximately the same thing they should do it the same way. If you cannot get a PT_LOAD section, maybe we can put a simple system in NOTE, or just create a PT_LOAD16 if the linker accepts other values. I do not really like to relocate after vmlinux has been linked at a fixed address, because I am not sure you can guess each address to relocate or not: you can define permanent address in the linker file by simply "symbol = address" and those should never be relocated. For instance the very high addresses on ia32 may point to registers or FPU instead of memory, so may or may not have to be relocated. I also would better like you not to relocate the real-mode addresses. > > And you do not want to write protect the kernel code (if the CPU write > > protection > > is not working, the hardware is not working so debug will be difficult, and > > a simple > > CRC32 can tell kernel memory failure) and use twice the same code memory > > (with different data area or saving kernel data elsewhere before reload). > > Is that related to module loading or instruction set detection/patch or > > multiprocessor? > > If I understand it right, you seem to be suggesting that I don't have to > reload the kernel text and I can only reload the data for second kernel? > > We run the whole of the kernel from a mutually execlusive location from > first kernel to mitigate the concerns that first kernel's ongoing DMA might > corrupt second kernel. That's why first kernel's text can't be reused. So in some exception handler, you detect something is wrong and then jump to a new kernel. Maybe this class of bug (DMA or hardware bit flipped) should be detected even without this double kernel environment, by running a CRC32 on the kernel text section in this exception handler, displayed that in the crash dump. > Secondly, it gives flexibility to user that either he can choose to use > the production kernel as capture kernel or an entirely different custom > kernel can be used as capture kernel. IHMO if it is just capturing the memory, I would have a kernel without any modules (usb-storage drivers linked-in) and save to USB key the core. Automatically loading modules after the crash may be a problem. > If real mode code is linked with vmlinux, then kdump will be broken. I do not want to break anything for fun. I need either a reason or a bug. > bzImage is relocatable. If a new kernel image format is introduced > (compressed ELF), then I will prefer it to be a relocatable one > (if possible). It may be possible to do some kind of "ld -r -o vmlinux", but delaying the decision where to run the kernel is just delaying: someone has to decide, and even if Gujin do not decide It will ask the user who most of the time will have no clue... and on which basis do you want the bootloader to decide... maybe this SDRAM is faulty and usually the SDRAM are 512 Mbytes so run the kernel at 512 + 16 Mbytes, but only if there is more than 1024 Mbytes of SDRAM? Thanks, Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Vivek Goyal <[EMAIL PROTECTED]> writes: > Kexec as a boot loader allows to load both ELF vmlinux file or bzImage. > Hence for kdump, a user got the flexibility to either use vmlinux or > bzImage for dump captruing purposes. Hence I am concerned about both. > > If real mode code is linked with vmlinux, then kdump will be broken. > > bzImage is relocatable. If a new kernel image format is introduced > (compressed ELF), then I will prefer it to be a relocatable one > (if possible). If we do make the real mode code findable from a set of ELF headers instead of a weird program header it should be an ELF note in the note segment that tells us where to find or possibly contains the real mode code. Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
On Thu, Feb 08, 2007 at 09:48:31AM +, Etienne Lorrain wrote: > Vivek Goyal wrote: > > > Etienne Lorrain wrote: > > > Yes, any PT_LOAD below 64 Kbytes can only be real mode, and real-mode > > > cannot be loaded higher, and cannot be bigger than 640 Kbytes, anything > > > different (like with virtual address at 0xC000) is Linux protected > > > mode. > > > Considering the linker used it is always the 4th program header, before > > > there > > > were only 3 program header,third one stay the NOTE one. > > > > Well this is all a lot of speculation. This is not standard way of > > retrieving > > information from ELF. Number of program headers finally created keep on > > changing. Previously it was left to the linker and now people have changed > > it to 3 by specifically using PHDR directive. > > Well, the standard way is: > 1 - the code program header, to be write protected if possible > 2 - the (initialised) data program header, to be able to restart the program > by just reload/re-entry, > and after that (memsiz - filesz) of BSS > Anything after is non standard. The number of program header is written in > the ELF header, > but anything over 2 probably need a special loader for the extra treatment. > That isn't usual to have something near zero, because it is better not to map > any memory there to catch null pointer dereference (in virtual memory system). > May be that's the ideal scenario. But if you leave it to linker to decide the number of progaram headers to linker then linker sometimes optimizes and creates more program headers if it finds empty space between various sections. Sometime one or two page space gets created between sections because of alignment restrictions. So effectively, number of program headers were not fixed, till recently. > I assume you care about this ELF header because you are also a user of > the ELF file vmlinux, aren't you? > Yes I am. I use kexec boot loader which has capability to load ELF kernel images (vmlinux). That's why I am concerned about linking real mode code in vmlinux as for kdump case I shall have to be aware that kernel vmlinux might contain a special PT_LOAD type program header which will contain real mode code and it does not have to be loaded. Then I will run into guessing business which one is that real mode PT_LOAD program header and my assumption might very well break in next few kernel release. > > > Just I did not completely understand why you need relocation (and the > > > announce > > > was when I was in holidays far away). I know a simple kernel is needed to > > > do > > > some debugging when the main kernel has crashed, this kernel is better > > > loaded > > > at for instance 16 Mbytes. You probably do not want the same kernel as > > > the main > > > one because it may crash the same way, and you start a loop - and even > > > then > > > if you have exactly the same kernel it is easier to use the same > > > write-protected > > > block of memory with different data sections. > > > But I probably do not understand the problem so do not know what to > > > write. > > > > As long as one can make sure that test kernel boots (commonly the case with > > distros), same kernel can be used as capture kernel too. So the idea here > > is to be able to use same kernel binary as production kernel and capture > > kernel hence distros don't have to ship an additional kernel binary compiled > > for a different physical addr just for dump capturing purposes. > > And you do not want to write protect the kernel code (if the CPU write > protection > is not working, the hardware is not working so debug will be difficult, and a > simple > CRC32 can tell kernel memory failure) and use twice the same code memory > (with different data area or saving kernel data elsewhere before reload). > Is that related to module loading or instruction set detection/patch or > multiprocessor? > If I understand it right, you seem to be suggesting that I don't have to reload the kernel text and I can only reload the data for second kernel? We run the whole of the kernel from a mutually execlusive location from first kernel to mitigate the concerns that first kernel's ongoing DMA might corrupt second kernel. That's why first kernel's text can't be reused. Secondly, it gives flexibility to user that either he can choose to use the production kernel as capture kernel or an entirely different custom kernel can be used as capture kernel. > > Currently relocation information is extracted from vmlinux and packed in > > final bzImage after some processing. After execution of real mode code > > and once the image is decompressed, all the relocations are performed and > > then control is transferred to kernel. > > So here you are not really using the initial ELF program header of vmlinux, > but more the section header and my PT_LOAD section bother you, I better > understand. You cannot really claim you are only doing standard/usual > ELF
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain <[EMAIL PROTECTED]> writes: >> Currently relocation information is extracted from vmlinux and packed in >> final bzImage after some processing. After execution of real mode code >> and once the image is decompressed, all the relocations are performed and >> then control is transferred to kernel. > > So here you are not really using the initial ELF program header of vmlinux, > but more the section header and my PT_LOAD section bother you, I better > understand. You cannot really claim you are only doing standard/usual > ELF treatment. The difference is that a boot loader is not doing the work. The format is. Compiling -fPIC almost gets us there except ld has bugs processing absolute symbols. Anything we expect a bootloader to do need to be as close as possible to the usual ELF treatment. Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Vivek Goyal wrote: > > Etienne Lorrain wrote: > > Yes, any PT_LOAD below 64 Kbytes can only be real mode, and real-mode > > cannot be loaded higher, and cannot be bigger than 640 Kbytes, anything > > different (like with virtual address at 0xC000) is Linux protected mode. > > Considering the linker used it is always the 4th program header, before > > there > > were only 3 program header,third one stay the NOTE one. > > Well this is all a lot of speculation. This is not standard way of retrieving > information from ELF. Number of program headers finally created keep on > changing. Previously it was left to the linker and now people have changed > it to 3 by specifically using PHDR directive. Well, the standard way is: 1 - the code program header, to be write protected if possible 2 - the (initialised) data program header, to be able to restart the program by just reload/re-entry, and after that (memsiz - filesz) of BSS Anything after is non standard. The number of program header is written in the ELF header, but anything over 2 probably need a special loader for the extra treatment. That isn't usual to have something near zero, because it is better not to map any memory there to catch null pointer dereference (in virtual memory system). I assume you care about this ELF header because you are also a user of the ELF file vmlinux, aren't you? > > Just I did not completely understand why you need relocation (and the > > announce > > was when I was in holidays far away). I know a simple kernel is needed to do > > some debugging when the main kernel has crashed, this kernel is better > > loaded > > at for instance 16 Mbytes. You probably do not want the same kernel as the > > main > > one because it may crash the same way, and you start a loop - and even then > > if you have exactly the same kernel it is easier to use the same > > write-protected > > block of memory with different data sections. > > But I probably do not understand the problem so do not know what to write. > > As long as one can make sure that test kernel boots (commonly the case with > distros), same kernel can be used as capture kernel too. So the idea here > is to be able to use same kernel binary as production kernel and capture > kernel hence distros don't have to ship an additional kernel binary compiled > for a different physical addr just for dump capturing purposes. And you do not want to write protect the kernel code (if the CPU write protection is not working, the hardware is not working so debug will be difficult, and a simple CRC32 can tell kernel memory failure) and use twice the same code memory (with different data area or saving kernel data elsewhere before reload). Is that related to module loading or instruction set detection/patch or multiprocessor? > Currently relocation information is extracted from vmlinux and packed in > final bzImage after some processing. After execution of real mode code > and once the image is decompressed, all the relocations are performed and > then control is transferred to kernel. So here you are not really using the initial ELF program header of vmlinux, but more the section header and my PT_LOAD section bother you, I better understand. You cannot really claim you are only doing standard/usual ELF treatment. Thanks, Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Vivek Goyal wrote: Etienne Lorrain wrote: Yes, any PT_LOAD below 64 Kbytes can only be real mode, and real-mode cannot be loaded higher, and cannot be bigger than 640 Kbytes, anything different (like with virtual address at 0xC000) is Linux protected mode. Considering the linker used it is always the 4th program header, before there were only 3 program header,third one stay the NOTE one. Well this is all a lot of speculation. This is not standard way of retrieving information from ELF. Number of program headers finally created keep on changing. Previously it was left to the linker and now people have changed it to 3 by specifically using PHDR directive. Well, the standard way is: 1 - the code program header, to be write protected if possible 2 - the (initialised) data program header, to be able to restart the program by just reload/re-entry, and after that (memsiz - filesz) of BSS Anything after is non standard. The number of program header is written in the ELF header, but anything over 2 probably need a special loader for the extra treatment. That isn't usual to have something near zero, because it is better not to map any memory there to catch null pointer dereference (in virtual memory system). I assume you care about this ELF header because you are also a user of the ELF file vmlinux, aren't you? Just I did not completely understand why you need relocation (and the announce was when I was in holidays far away). I know a simple kernel is needed to do some debugging when the main kernel has crashed, this kernel is better loaded at for instance 16 Mbytes. You probably do not want the same kernel as the main one because it may crash the same way, and you start a loop - and even then if you have exactly the same kernel it is easier to use the same write-protected block of memory with different data sections. But I probably do not understand the problem so do not know what to write. As long as one can make sure that test kernel boots (commonly the case with distros), same kernel can be used as capture kernel too. So the idea here is to be able to use same kernel binary as production kernel and capture kernel hence distros don't have to ship an additional kernel binary compiled for a different physical addr just for dump capturing purposes. And you do not want to write protect the kernel code (if the CPU write protection is not working, the hardware is not working so debug will be difficult, and a simple CRC32 can tell kernel memory failure) and use twice the same code memory (with different data area or saving kernel data elsewhere before reload). Is that related to module loading or instruction set detection/patch or multiprocessor? Currently relocation information is extracted from vmlinux and packed in final bzImage after some processing. After execution of real mode code and once the image is decompressed, all the relocations are performed and then control is transferred to kernel. So here you are not really using the initial ELF program header of vmlinux, but more the section header and my PT_LOAD section bother you, I better understand. You cannot really claim you are only doing standard/usual ELF treatment. Thanks, Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain [EMAIL PROTECTED] writes: Currently relocation information is extracted from vmlinux and packed in final bzImage after some processing. After execution of real mode code and once the image is decompressed, all the relocations are performed and then control is transferred to kernel. So here you are not really using the initial ELF program header of vmlinux, but more the section header and my PT_LOAD section bother you, I better understand. You cannot really claim you are only doing standard/usual ELF treatment. The difference is that a boot loader is not doing the work. The format is. Compiling -fPIC almost gets us there except ld has bugs processing absolute symbols. Anything we expect a bootloader to do need to be as close as possible to the usual ELF treatment. Eric - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
On Thu, Feb 08, 2007 at 09:48:31AM +, Etienne Lorrain wrote: Vivek Goyal wrote: Etienne Lorrain wrote: Yes, any PT_LOAD below 64 Kbytes can only be real mode, and real-mode cannot be loaded higher, and cannot be bigger than 640 Kbytes, anything different (like with virtual address at 0xC000) is Linux protected mode. Considering the linker used it is always the 4th program header, before there were only 3 program header,third one stay the NOTE one. Well this is all a lot of speculation. This is not standard way of retrieving information from ELF. Number of program headers finally created keep on changing. Previously it was left to the linker and now people have changed it to 3 by specifically using PHDR directive. Well, the standard way is: 1 - the code program header, to be write protected if possible 2 - the (initialised) data program header, to be able to restart the program by just reload/re-entry, and after that (memsiz - filesz) of BSS Anything after is non standard. The number of program header is written in the ELF header, but anything over 2 probably need a special loader for the extra treatment. That isn't usual to have something near zero, because it is better not to map any memory there to catch null pointer dereference (in virtual memory system). May be that's the ideal scenario. But if you leave it to linker to decide the number of progaram headers to linker then linker sometimes optimizes and creates more program headers if it finds empty space between various sections. Sometime one or two page space gets created between sections because of alignment restrictions. So effectively, number of program headers were not fixed, till recently. I assume you care about this ELF header because you are also a user of the ELF file vmlinux, aren't you? Yes I am. I use kexec boot loader which has capability to load ELF kernel images (vmlinux). That's why I am concerned about linking real mode code in vmlinux as for kdump case I shall have to be aware that kernel vmlinux might contain a special PT_LOAD type program header which will contain real mode code and it does not have to be loaded. Then I will run into guessing business which one is that real mode PT_LOAD program header and my assumption might very well break in next few kernel release. Just I did not completely understand why you need relocation (and the announce was when I was in holidays far away). I know a simple kernel is needed to do some debugging when the main kernel has crashed, this kernel is better loaded at for instance 16 Mbytes. You probably do not want the same kernel as the main one because it may crash the same way, and you start a loop - and even then if you have exactly the same kernel it is easier to use the same write-protected block of memory with different data sections. But I probably do not understand the problem so do not know what to write. As long as one can make sure that test kernel boots (commonly the case with distros), same kernel can be used as capture kernel too. So the idea here is to be able to use same kernel binary as production kernel and capture kernel hence distros don't have to ship an additional kernel binary compiled for a different physical addr just for dump capturing purposes. And you do not want to write protect the kernel code (if the CPU write protection is not working, the hardware is not working so debug will be difficult, and a simple CRC32 can tell kernel memory failure) and use twice the same code memory (with different data area or saving kernel data elsewhere before reload). Is that related to module loading or instruction set detection/patch or multiprocessor? If I understand it right, you seem to be suggesting that I don't have to reload the kernel text and I can only reload the data for second kernel? We run the whole of the kernel from a mutually execlusive location from first kernel to mitigate the concerns that first kernel's ongoing DMA might corrupt second kernel. That's why first kernel's text can't be reused. Secondly, it gives flexibility to user that either he can choose to use the production kernel as capture kernel or an entirely different custom kernel can be used as capture kernel. Currently relocation information is extracted from vmlinux and packed in final bzImage after some processing. After execution of real mode code and once the image is decompressed, all the relocations are performed and then control is transferred to kernel. So here you are not really using the initial ELF program header of vmlinux, but more the section header and my PT_LOAD section bother you, I better understand. You cannot really claim you are only doing standard/usual ELF treatment. Kexec as a boot loader allows to load both ELF vmlinux file or bzImage. Hence for kdump, a user got the flexibility to
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Vivek Goyal [EMAIL PROTECTED] writes: Kexec as a boot loader allows to load both ELF vmlinux file or bzImage. Hence for kdump, a user got the flexibility to either use vmlinux or bzImage for dump captruing purposes. Hence I am concerned about both. If real mode code is linked with vmlinux, then kdump will be broken. bzImage is relocatable. If a new kernel image format is introduced (compressed ELF), then I will prefer it to be a relocatable one (if possible). If we do make the real mode code findable from a set of ELF headers instead of a weird program header it should be an ELF note in the note segment that tells us where to find or possibly contains the real mode code. Eric - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
I assume you care about this ELF header because you are also a user of the ELF file vmlinux, aren't you? Yes I am. I use kexec boot loader which has capability to load ELF kernel images (vmlinux). That's why I am concerned about linking real mode code in vmlinux as for kdump case I shall have to be aware that kernel vmlinux might contain a special PT_LOAD type program header which will contain real mode code and it does not have to be loaded. Then I will run into guessing business which one is that real mode PT_LOAD program header and my assumption might very well break in next few kernel release. So is kexec able to relocate this vmlinux? If kexec and Gujin do approximately the same thing they should do it the same way. If you cannot get a PT_LOAD section, maybe we can put a simple system in NOTE, or just create a PT_LOAD16 if the linker accepts other values. I do not really like to relocate after vmlinux has been linked at a fixed address, because I am not sure you can guess each address to relocate or not: you can define permanent address in the linker file by simply symbol = address and those should never be relocated. For instance the very high addresses on ia32 may point to registers or FPU instead of memory, so may or may not have to be relocated. I also would better like you not to relocate the real-mode addresses. And you do not want to write protect the kernel code (if the CPU write protection is not working, the hardware is not working so debug will be difficult, and a simple CRC32 can tell kernel memory failure) and use twice the same code memory (with different data area or saving kernel data elsewhere before reload). Is that related to module loading or instruction set detection/patch or multiprocessor? If I understand it right, you seem to be suggesting that I don't have to reload the kernel text and I can only reload the data for second kernel? We run the whole of the kernel from a mutually execlusive location from first kernel to mitigate the concerns that first kernel's ongoing DMA might corrupt second kernel. That's why first kernel's text can't be reused. So in some exception handler, you detect something is wrong and then jump to a new kernel. Maybe this class of bug (DMA or hardware bit flipped) should be detected even without this double kernel environment, by running a CRC32 on the kernel text section in this exception handler, displayed that in the crash dump. Secondly, it gives flexibility to user that either he can choose to use the production kernel as capture kernel or an entirely different custom kernel can be used as capture kernel. IHMO if it is just capturing the memory, I would have a kernel without any modules (usb-storage drivers linked-in) and save to USB key the core. Automatically loading modules after the crash may be a problem. If real mode code is linked with vmlinux, then kdump will be broken. I do not want to break anything for fun. I need either a reason or a bug. bzImage is relocatable. If a new kernel image format is introduced (compressed ELF), then I will prefer it to be a relocatable one (if possible). It may be possible to do some kind of ld -r -o vmlinux, but delaying the decision where to run the kernel is just delaying: someone has to decide, and even if Gujin do not decide It will ask the user who most of the time will have no clue... and on which basis do you want the bootloader to decide... maybe this SDRAM is faulty and usually the SDRAM are 512 Mbytes so run the kernel at 512 + 16 Mbytes, but only if there is more than 1024 Mbytes of SDRAM? Thanks, Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
On Thu, Feb 08, 2007 at 01:10:58PM +, Etienne Lorrain wrote: I assume you care about this ELF header because you are also a user of the ELF file vmlinux, aren't you? Yes I am. I use kexec boot loader which has capability to load ELF kernel images (vmlinux). That's why I am concerned about linking real mode code in vmlinux as for kdump case I shall have to be aware that kernel vmlinux might contain a special PT_LOAD type program header which will contain real mode code and it does not have to be loaded. Then I will run into guessing business which one is that real mode PT_LOAD program header and my assumption might very well break in next few kernel release. So is kexec able to relocate this vmlinux? If kexec and Gujin do approximately the same thing they should do it the same way. No, as of today, kexec does not relocate vmlinux. At compilation time, vmlinux is compiled for a fixed address and vmlinux is loaded at that address. This compile address can be controlled with CONFIG_PHYSICAL_START, as you already mentioned in your patchset. So in the case of kdump, if one wishes to use vmlinux instead of a relocatable bzImage, he will re-compile the kernel for a different address and then use that vmlinux. OTOH, boot loader can possibly relocate vmlinux because all the relocation information is retained in vmlinux (Using ld option --emit-relocs). But probably self relocating image should be a better option as all the boot-loaders out there don't have to be modifed. If you cannot get a PT_LOAD section, maybe we can put a simple system in NOTE, or just create a PT_LOAD16 if the linker accepts other values. My guess is that PT_LOAD16 is not an acceptable value. Putting information in PT_NOTE seems interesting (As Eric already mentioned). I do not really like to relocate after vmlinux has been linked at a fixed address, because I am not sure you can guess each address to relocate or not: you can define permanent address in the linker file by simply symbol = address and those should never be relocated. For instance the very high addresses on ia32 may point to registers or FPU instead of memory, so may or may not have to be relocated. I also would better like you not to relocate the real-mode addresses. I think there will have to be an upper limit on the address where you can relocate your image to. Anyway, given i386 virtual memory address space constraint, I think you can not load kernel image in high memory (896MB). It has to run anywhere below that. I think you are referring to defining absolute symbols. Already there are some absolute symbols generated when kernel is compiled and these are not relocated while image is being loaded at a non-compiled address. So if a boot loader decides to relocate a vmlinux image, it will have to skip relocations present w.r.t absolute symbols. (Unfortunately, ld does generate relocation entries even for absolute symbols). And you do not want to write protect the kernel code (if the CPU write protection is not working, the hardware is not working so debug will be difficult, and a simple CRC32 can tell kernel memory failure) and use twice the same code memory (with different data area or saving kernel data elsewhere before reload). Is that related to module loading or instruction set detection/patch or multiprocessor? If I understand it right, you seem to be suggesting that I don't have to reload the kernel text and I can only reload the data for second kernel? We run the whole of the kernel from a mutually execlusive location from first kernel to mitigate the concerns that first kernel's ongoing DMA might corrupt second kernel. That's why first kernel's text can't be reused. So in some exception handler, you detect something is wrong and then jump to a new kernel. In a nutshell, Yes. Maybe this class of bug (DMA or hardware bit flipped) should be detected even without this double kernel environment, by running a CRC32 on the kernel text section in this exception handler, displayed that in the crash dump. We already do checksum verification to make sure newly loaded kernel has not been corrupted before we jump to it. What's the point in doing the verification existing kernel text. Because even that is intact, and you reload your data section, that data section will have to be reloaded at the same place where previous data section was. Now after you re-loaded the data section, some DMA might corrupt it now (Because we never stopped DMAs). So loading a fresh copy of text and data section at a reserved memory location seems safer. Secondly, it gives flexibility to user that either he can choose to use the production kernel as capture kernel or an entirely different custom kernel can be used as capture kernel. IHMO if it is just capturing the memory, I would have a kernel without any modules (usb-storage drivers linked-in) and save to USB
Re: Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
On Wed, Feb 07, 2007 at 02:55:39PM +, Etienne Lorrain wrote: > Vivek Goyal wrote: > >How do I know which program header is real mode code and the boot loader > > is not supposed to load it? May be PT_LOAD header with physical addr 0? > > What happens if changes happen and down the line we start compiling > > real mode code for non-zero address? > > Yes, any PT_LOAD below 64 Kbytes can only be real mode, and real-mode > cannot be loaded higher, and cannot be bigger than 640 Kbytes, anything > different (like with virtual address at 0xC000) is Linux protected mode. > Considering the linker used it is always the 4th program header, before there > were only 3 program header,third one stay the NOTE one. > Well this is all a lot of speculation. This is not standard way of retrieving information from ELF. Number of program headers finally created keep on changing. Previously it was left to the linker and now people have changed it to 3 by specifically using PHDR directive. > > Hence I think keeping real mode code out of vmlinux might prove to be a > > good idea. > > Well, ELF is a format made to describe what to load, at which addres, and I > assume > everyone recognise that some ELF section shall not be loaded, like the debug > information. > Gujin accepts having the real-mode treated like a program text and so in the > program header, > or like extra memory block with content like the symbol table used by a > debugger, I do not > know what is best so have implemented the two. > > > Secondly, if you compile real mode code with vmlinux, what would be the > > entry point for this ELF file? Real mode entry? Then I have not way to > > find out from ELF headers where is the protected mode entry point and > > I can not do use this vmlinux with kexec/kdump. > > I did not touch the kernel entry point, the real mode entry point is at > offset zero > in the real-mode program header/section table. > > > OTOH, now bzImage is relocatable. Is this image going to be relocatable? > > How do we take care of that? > > Just I did not completely understand why you need relocation (and the > announce > was when I was in holidays far away). I know a simple kernel is needed to do > some debugging when the main kernel has crashed, this kernel is better loaded > at for instance 16 Mbytes. You probably do not want the same kernel as the > main > one because it may crash the same way, and you start a loop - and even then > if you have exactly the same kernel it is easier to use the same > write-protected > block of memory with different data sections. > But I probably do not understand the problem so do not know what to write. > As long as one can make sure that test kernel boots (commonly the case with distros), same kernel can be used as capture kernel too. So the idea here is to be able to use same kernel binary as production kernel and capture kernel hence distros don't have to ship an additional kernel binary compiled for a different physical addr just for dump capturing purposes. Currently relocation information is extracted from vmlinux and packed in final bzImage after some processing. After execution of real mode code and once the image is decompressed, all the relocations are performed and then control is transferred to kernel. Thanks Vivek - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain wrote: I did try to remove by "sed" some structure easily converted, like: mov 26(%ebx),%eax by: mov 26(%bx),%eax for all registers esi,edi,ebx,ebp but it saved so few bytes that I did not keep it. Well, *of course*, since you still do all your pointer arithmetic in 32-bit space, plus you can't convert all 32-bit modes this way. This is a far cry from a proper port. -hpa - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Vivek Goyal wrote: >How do I know which program header is real mode code and the boot loader > is not supposed to load it? May be PT_LOAD header with physical addr 0? > What happens if changes happen and down the line we start compiling > real mode code for non-zero address? Yes, any PT_LOAD below 64 Kbytes can only be real mode, and real-mode cannot be loaded higher, and cannot be bigger than 640 Kbytes, anything different (like with virtual address at 0xC000) is Linux protected mode. Considering the linker used it is always the 4th program header, before there were only 3 program header,third one stay the NOTE one. > Hence I think keeping real mode code out of vmlinux might prove to be a good > idea. Well, ELF is a format made to describe what to load, at which addres, and I assume everyone recognise that some ELF section shall not be loaded, like the debug information. Gujin accepts having the real-mode treated like a program text and so in the program header, or like extra memory block with content like the symbol table used by a debugger, I do not know what is best so have implemented the two. > Secondly, if you compile real mode code with vmlinux, what would be the > entry point for this ELF file? Real mode entry? Then I have not way to > find out from ELF headers where is the protected mode entry point and > I can not do use this vmlinux with kexec/kdump. I did not touch the kernel entry point, the real mode entry point is at offset zero in the real-mode program header/section table. > OTOH, now bzImage is relocatable. Is this image going to be relocatable? > How do we take care of that? Just I did not completely understand why you need relocation (and the announce was when I was in holidays far away). I know a simple kernel is needed to do some debugging when the main kernel has crashed, this kernel is better loaded at for instance 16 Mbytes. You probably do not want the same kernel as the main one because it may crash the same way, and you start a loop - and even then if you have exactly the same kernel it is easier to use the same write-protected block of memory with different data sections. But I probably do not understand the problem so do not know what to write. Thanks, Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Eric W. Biederman wote: > H. Peter Anvin wrote: > > That's not a valid comparison, because you're using 32-bit registers to hold > > 16-bit pointers, and then sticking 67 prefixes on completely unnecessarily. > > Regardless if the size is good enough we can use it :) > > For romcc I had a 3x code bloat over had crafted assembler, because I > had no memory and had to inline every single function call because I > didn't have someplace to put a return register. So I have the > scenario with the most code bloat. I win! It is true that I try to put all my variables in big structures in Gujin, not only because I think it is cleaner (when variables are related in their use) but also because it reduces the problem of 32 bits addresses by using offsets to the structure pointer, most of the time. In some cases, using 32 bits address prefix is better because offsets to ESP are possible but in 16 bits you first have to copy SP to BP to reference stack parameters. Also you can use any register as a pointer. Anyway, we are currently talking of 6-7 Kbytes of real-mode section for the ELF, including root autodetection and command line generation - there is a lot of source code to describe all those structures but no "code bloat". I did try to remove by "sed" some structure easily converted, like: mov 26(%ebx),%eax by: mov 26(%bx),%eax for all registers esi,edi,ebx,ebp but it saved so few bytes that I did not keep it. Eric W. Biederman wote: > This code is currently completely Gujin specific. If concieved as a > replacement for setup.S it has a chance of passing review. If someone ever want to transfer some code from Gujin to Linux, without reducing too much the functionalities, I would be more than happy about it - I could even try to help - but considering that I even do not have enough time to timely answer E-mails (considering my paying job) and I have a lot of other parts of Gujin to improve, I can not lead at all on this project. I am not sure this task is even possible, I do not know where should be the limit in between bootloader and kernel - considering that there is also the initrd (why people didn't switch to initramfs already?), and being able to debug the real-mode code (like those DBG*.exe Gujin files), return error code, check CRC32 is a minimum. > LILO is a lot saner then Grub, and it still supports more filesystems... > Just because it memorizes it all before you shut down the system for > simplicity doesn't mean lilo is worse. LILO is quite good, but I cannot talk too much of other bootloader because I simply do not use them enough (excluding ISOLINUX). At least LILO do not switch too early to protected mode, like Gujin. The thing I do not like about it is that there isn't much checks done, like a CRC32, so you may load the right sectors from the wrong disk. Also, "ioctl (file_desc, FIBMAP, )" seem to be less and less supported, either by filesystems themselves of by lower layers like LVM and the like. The standalone Linux executable "showmap" in Gujin show problems sometimes. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Eric W. Biederman wote: H. Peter Anvin wrote: That's not a valid comparison, because you're using 32-bit registers to hold 16-bit pointers, and then sticking 67 prefixes on completely unnecessarily. Regardless if the size is good enough we can use it :) For romcc I had a 3x code bloat over had crafted assembler, because I had no memory and had to inline every single function call because I didn't have someplace to put a return register. So I have the scenario with the most code bloat. I win! It is true that I try to put all my variables in big structures in Gujin, not only because I think it is cleaner (when variables are related in their use) but also because it reduces the problem of 32 bits addresses by using offsets to the structure pointer, most of the time. In some cases, using 32 bits address prefix is better because offsets to ESP are possible but in 16 bits you first have to copy SP to BP to reference stack parameters. Also you can use any register as a pointer. Anyway, we are currently talking of 6-7 Kbytes of real-mode section for the ELF, including root autodetection and command line generation - there is a lot of source code to describe all those structures but no code bloat. I did try to remove by sed some structure easily converted, like: mov 26(%ebx),%eax by: mov 26(%bx),%eax for all registers esi,edi,ebx,ebp but it saved so few bytes that I did not keep it. Eric W. Biederman wote: This code is currently completely Gujin specific. If concieved as a replacement for setup.S it has a chance of passing review. If someone ever want to transfer some code from Gujin to Linux, without reducing too much the functionalities, I would be more than happy about it - I could even try to help - but considering that I even do not have enough time to timely answer E-mails (considering my paying job) and I have a lot of other parts of Gujin to improve, I can not lead at all on this project. I am not sure this task is even possible, I do not know where should be the limit in between bootloader and kernel - considering that there is also the initrd (why people didn't switch to initramfs already?), and being able to debug the real-mode code (like those DBG*.exe Gujin files), return error code, check CRC32 is a minimum. LILO is a lot saner then Grub, and it still supports more filesystems... Just because it memorizes it all before you shut down the system for simplicity doesn't mean lilo is worse. LILO is quite good, but I cannot talk too much of other bootloader because I simply do not use them enough (excluding ISOLINUX). At least LILO do not switch too early to protected mode, like Gujin. The thing I do not like about it is that there isn't much checks done, like a CRC32, so you may load the right sectors from the wrong disk. Also, ioctl (file_desc, FIBMAP, block) seem to be less and less supported, either by filesystems themselves of by lower layers like LVM and the like. The standalone Linux executable showmap in Gujin show problems sometimes. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Vivek Goyal wrote: How do I know which program header is real mode code and the boot loader is not supposed to load it? May be PT_LOAD header with physical addr 0? What happens if changes happen and down the line we start compiling real mode code for non-zero address? Yes, any PT_LOAD below 64 Kbytes can only be real mode, and real-mode cannot be loaded higher, and cannot be bigger than 640 Kbytes, anything different (like with virtual address at 0xC000) is Linux protected mode. Considering the linker used it is always the 4th program header, before there were only 3 program header,third one stay the NOTE one. Hence I think keeping real mode code out of vmlinux might prove to be a good idea. Well, ELF is a format made to describe what to load, at which addres, and I assume everyone recognise that some ELF section shall not be loaded, like the debug information. Gujin accepts having the real-mode treated like a program text and so in the program header, or like extra memory block with content like the symbol table used by a debugger, I do not know what is best so have implemented the two. Secondly, if you compile real mode code with vmlinux, what would be the entry point for this ELF file? Real mode entry? Then I have not way to find out from ELF headers where is the protected mode entry point and I can not do use this vmlinux with kexec/kdump. I did not touch the kernel entry point, the real mode entry point is at offset zero in the real-mode program header/section table. OTOH, now bzImage is relocatable. Is this image going to be relocatable? How do we take care of that? Just I did not completely understand why you need relocation (and the announce was when I was in holidays far away). I know a simple kernel is needed to do some debugging when the main kernel has crashed, this kernel is better loaded at for instance 16 Mbytes. You probably do not want the same kernel as the main one because it may crash the same way, and you start a loop - and even then if you have exactly the same kernel it is easier to use the same write-protected block of memory with different data sections. But I probably do not understand the problem so do not know what to write. Thanks, Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain wrote: I did try to remove by sed some structure easily converted, like: mov 26(%ebx),%eax by: mov 26(%bx),%eax for all registers esi,edi,ebx,ebp but it saved so few bytes that I did not keep it. Well, *of course*, since you still do all your pointer arithmetic in 32-bit space, plus you can't convert all 32-bit modes this way. This is a far cry from a proper port. -hpa - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
On Wed, Feb 07, 2007 at 02:55:39PM +, Etienne Lorrain wrote: Vivek Goyal wrote: How do I know which program header is real mode code and the boot loader is not supposed to load it? May be PT_LOAD header with physical addr 0? What happens if changes happen and down the line we start compiling real mode code for non-zero address? Yes, any PT_LOAD below 64 Kbytes can only be real mode, and real-mode cannot be loaded higher, and cannot be bigger than 640 Kbytes, anything different (like with virtual address at 0xC000) is Linux protected mode. Considering the linker used it is always the 4th program header, before there were only 3 program header,third one stay the NOTE one. Well this is all a lot of speculation. This is not standard way of retrieving information from ELF. Number of program headers finally created keep on changing. Previously it was left to the linker and now people have changed it to 3 by specifically using PHDR directive. Hence I think keeping real mode code out of vmlinux might prove to be a good idea. Well, ELF is a format made to describe what to load, at which addres, and I assume everyone recognise that some ELF section shall not be loaded, like the debug information. Gujin accepts having the real-mode treated like a program text and so in the program header, or like extra memory block with content like the symbol table used by a debugger, I do not know what is best so have implemented the two. Secondly, if you compile real mode code with vmlinux, what would be the entry point for this ELF file? Real mode entry? Then I have not way to find out from ELF headers where is the protected mode entry point and I can not do use this vmlinux with kexec/kdump. I did not touch the kernel entry point, the real mode entry point is at offset zero in the real-mode program header/section table. OTOH, now bzImage is relocatable. Is this image going to be relocatable? How do we take care of that? Just I did not completely understand why you need relocation (and the announce was when I was in holidays far away). I know a simple kernel is needed to do some debugging when the main kernel has crashed, this kernel is better loaded at for instance 16 Mbytes. You probably do not want the same kernel as the main one because it may crash the same way, and you start a loop - and even then if you have exactly the same kernel it is easier to use the same write-protected block of memory with different data sections. But I probably do not understand the problem so do not know what to write. As long as one can make sure that test kernel boots (commonly the case with distros), same kernel can be used as capture kernel too. So the idea here is to be able to use same kernel binary as production kernel and capture kernel hence distros don't have to ship an additional kernel binary compiled for a different physical addr just for dump capturing purposes. Currently relocation information is extracted from vmlinux and packed in final bzImage after some processing. After execution of real mode code and once the image is decompressed, all the relocations are performed and then control is transferred to kernel. Thanks Vivek - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
On Tue, Feb 06, 2007 at 01:12:05PM +, Etienne Lorrain wrote: > > Building real mode code with kernel binary (vmlinux) has got another > > disadvantage that it breaks using vmlinux for kdump purposes. One compiles > > the kernel binary to execute from a different address but real mode > > code/data > > will continue to be at virtual/physical addr 0 and kexec can not load it > > as that physical memory is not available at all. Kdump skips the real mode > > code execution. > > But that is exactly what you want and need for kdump, isn't it? > The ELF file did not change, the program header has the last index at > address 0 that you do not want to load because you do not want to > execute the real-mode code. Load the rest and provide the 4 Kbytes > parameter page - it should work. How do I know which program header is real mode code and the boot loader is not supposed to load it? May be PT_LOAD header with physical addr 0? What happens if changes happen and down the line we start compiling real mode code for non-zero address? Hence I think keeping real mode code out of vmlinux might prove to be a good idea. Secondly, if you compile real mode code with vmlinux, what would be the entry point for this ELF file? Real mode entry? Then I have not way to find out from ELF headers where is the protected mode entry point and I can not do use this vmlinux with kexec/kdump. OTOH, now bzImage is relocatable. Is this image going to be relocatable? How do we take care of that? Thanks Vivek - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : Re : Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
"H. Peter Anvin" <[EMAIL PROTECTED]> writes: > Etienne Lorrain wrote: >> H. Peter Anvin wrote: >>> I've long wished that someone would do a proper 16-bit x86 port of gcc; >> >>> however, the .code16gcc is usually good enough, although it produces code >>> which is a lot bigger than it needs to be. >> >> It is only that much bigger if you compare to 16 bits integer compilers, >> but once you deal with 32 bits integers in real mode you have one extra byte >> out of the two - which is still better than dealing with dx:ax. >> >> Last time I compiled (long time ago) some piece of code with and without >> .code16gcc it was approx 15% bigger - approx what you get when optimising. >> > > That's not a valid comparison, because you're using 32-bit registers to hold > 16-bit pointers, and then sticking 67 prefixes on completely unnecessarily. Regardless if the size is good enough we can use it :) For romcc I had a 3x code bloat over had crafted assembler, because I had no memory and had to inline every single function call because I didn't have someplace to put a return register. So I have the scenario with the most code bloat. I win! Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : Re : Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain wrote: H. Peter Anvin wrote: I've long wished that someone would do a proper 16-bit x86 port of gcc; however, the .code16gcc is usually good enough, although it produces code which is a lot bigger than it needs to be. It is only that much bigger if you compare to 16 bits integer compilers, but once you deal with 32 bits integers in real mode you have one extra byte out of the two - which is still better than dealing with dx:ax. Last time I compiled (long time ago) some piece of code with and without .code16gcc it was approx 15% bigger - approx what you get when optimising. That's not a valid comparison, because you're using 32-bit registers to hold 16-bit pointers, and then sticking 67 prefixes on completely unnecessarily. -hpa - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : Re : Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
H. Peter Anvin wrote: > I've long wished that someone would do a proper 16-bit x86 port of gcc; > however, the .code16gcc is usually good enough, although it produces > code which is a lot bigger than it needs to be. It is only that much bigger if you compare to 16 bits integer compilers, but once you deal with 32 bits integers in real mode you have one extra byte out of the two - which is still better than dealing with dx:ax. Last time I compiled (long time ago) some piece of code with and without .code16gcc it was approx 15% bigger - approx what you get when optimising. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Eric W. Biederman wrote: Modifying the linux real mode assembler, nobody could even want to. I have several times. It's just code. But I do agree it will likely be more maintainable if we can convert it to C. On that same token I wrote a compiler in romcc in another context so I didn't have to write so much assembly. And for maintainability and comprehensibility it was a big help. I've long wished that someone would do a proper 16-bit x86 port of gcc; however, the .code16gcc is usually good enough, although it produces code which is a lot bigger than it needs to be. -hpa - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain <[EMAIL PROTECTED]> writes: > I am not sure to understand: Gujin calls a function inside the ELF real > mode program header of the Linux kernel. All the system is currently > in real mode. There isn't any limitation in this function of the kernel to > decide to print some message and not return at all, this function can > selectively do so by reading which bootloader ID it is using (another > parameter). It is not done mostly because "printf" is difficult to do > in assembler, Gujin has no problem. This code is currently completely Gujin specific. If concieved as a replacement for setup.S it has a chance of passing review. >> Is your real mode C code section relative such that it can be loaded >> at different addresses and still work? > > The code is relative - but not the data (first 4 Kbytes at %ds:0 and stack > available) - but the whole lot at any segment boundary, i.e. every 16 bytes. > Else Gujin would not work under DOS. Ok. Similar restrictions to the current real mode code. >> The program header is for executable loaders the section header is for >> linkers and the section header is optional in PT_LOAD and PT_DYN >> executables. Use of the section header by a loader is a bug. > > Unless if there is a problem, Gujin uses only the program header; > it has a look in the section header just in case - that can be removed > easily. I was wondering about a relocatable image there - but I do > not know enough for that. When you are relocatable no relocations are processed and all addresses in the program header are slid by some fixed amount (preserving alignment) after that the relocated object has to work out the relocations by itself. >> > Well, you can generate with the proposed patch and boot with SYSLINUX, >> > Grub and LILO, but trying to clean Linux real-mode code is where I begun. >> >> I haven't looked yet. But I believe this is something that we can do, >> so long as cleaning and feature enhancement are not mixed in a bad way. >> Phrasing this another way. What we can do with the interface is >> determined our interface to existing bootloaders and what bootloaders >> exist. Basically it is the rule that we need to preserve backwards >> compatibility and changes generally need to be evolutionary ones. > > Well, if you want to preserve compatibility with other bootloaders, > it is probably possible to put some source around the ELF kernel file, > mostly taken from Gujin if you want (GPL), but I wonder why you would > like to be compatible with LILO. LILO is a lot saner then Grub, and it still supports more filesystems... Just because it memorizes it all before you shut down the system for simplicity doesn't mean lilo is worse. > Modifying the linux real mode assembler, nobody could even want to. I have several times. It's just code. But I do agree it will likely be more maintainable if we can convert it to C. On that same token I wrote a compiler in romcc in another context so I didn't have to write so much assembly. And for maintainability and comprehensibility it was a big help. Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Eric W. Biederman wrote: > Etienne Lorrain <[EMAIL PROTECTED]> writes: >> Well, if the function called at offset 0 in the real-mode section return >> non zero, that is probably an error - that is Gujin interface, but do not >> ask >> me to change other bootloaders to handle that. >> This function is called with few parameters, one is a string pointer >> if the function do not want to print the problem report itself. >> The function is called with an offset of zero in its own intel >> segment. > > So I think there are some backwards compatibility issues with your > real mode code interface. Setting a new flag that says to return instead > of printing an error message and halting would be one way to handle > this. I am not sure to understand: Gujin calls a function inside the ELF real mode program header of the Linux kernel. All the system is currently in real mode. There isn't any limitation in this function of the kernel to decide to print some message and not return at all, this function can selectively do so by reading which bootloader ID it is using (another parameter). It is not done mostly because "printf" is difficult to do in assembler, Gujin has no problem. > Is your real mode C code section relative such that it can be loaded > at different addresses and still work? The code is relative - but not the data (first 4 Kbytes at %ds:0 and stack available) - but the whole lot at any segment boundary, i.e. every 16 bytes. Else Gujin would not work under DOS. > The program header is for executable loaders the section header is for > linkers and the section header is optional in PT_LOAD and PT_DYN > executables. Use of the section header by a loader is a bug. Unless if there is a problem, Gujin uses only the program header; it has a look in the section header just in case - that can be removed easily. I was wondering about a relocatable image there - but I do not know enough for that. > There have been limitations but mostly with respect to page table size > and the like but they were not limitations a bootloader had to deal with. Was talking about this comment, but it is old: /* 0x28000*16 = 2.5 MB, conservative estimate for the current maximum */ http://lxr.linux.no/source/arch/i386/boot/tools/build.c?v=2.4.28 > > Well, you can generate with the proposed patch and boot with SYSLINUX, > > Grub and LILO, but trying to clean Linux real-mode code is where I begun. > > I haven't looked yet. But I believe this is something that we can do, > so long as cleaning and feature enhancement are not mixed in a bad way. > Phrasing this another way. What we can do with the interface is > determined our interface to existing bootloaders and what bootloaders > exist. Basically it is the rule that we need to preserve backwards > compatibility and changes generally need to be evolutionary ones. Well, if you want to preserve compatibility with other bootloaders, it is probably possible to put some source around the ELF kernel file, mostly taken from Gujin if you want (GPL), but I wonder why you would like to be compatible with LILO. Modifying the linux real mode assembler, nobody could even want to. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain <[EMAIL PROTECTED]> writes: > Eric W. Biederman wrote: >> Etienne Lorrain <[EMAIL PROTECTED]> writes: >> > New features are more that this real-mode function can return an error >> > to the bootloader to tell something to the user, so the user can select >> > another kernel with the right processor, another video mode... with >> > clean error messages - not a crash dump because this assembly >> > instruction is not for that processor. >> >> Having an error handling compatibility that is backwards compatible sounds >> interesting. > > Well, if the function called at offset 0 in the real-mode section return > non zero, that is probably an error - that is Gujin interface, but do not ask > me to change other bootloaders to handle that. > This function is called with few parameters, one is a string pointer > if the function do not want to print the problem report itself. > The function is called with an offset of zero in its own intel > segment. So I think there are some backwards compatibility issues with your real mode code interface. Setting a new flag that says to return instead of printing an error message and halting would be one way to handle this. Is your real mode C code section relative such that it can be loaded at different addresses and still work? >> > I am still saying that the bootloader knows the root filesystem to >> > be used by the kernel it loads, and that ELF is a clean format to >> > store different sections to be loaded into memory at predefined >> > addresses. >> >> Yes. Although when you think sections instead of segments I'm a little >> worried. > > Well, for my defense, if Gujin do not find the real mode code in the program > header it search for it in the section table, if the program header cannot be > modified for some reason. There may be some issues if that ELF file is > "stripped" but it seems to work on my side. The program header is for executable loaders the section header is for linkers and the section header is optional in PT_LOAD and PT_DYN executables. Use of the section header by a loader is a bug. >> > Also there isn't any more kernel size limit. >> >> I think as HPA points out we have gotten past that a long time >> ago with the bzImage format. > > There was still a test not that long ago (in the code I am not executing) - > but > I was wrong to bring this point forward. There have been limitations but mostly with respect to page table size and the like but they were not limitations a bootloader had to deal with. >> With the right delicacy, and preserving backwards compatibility >> with existing bootloaders I think we can achieve things. > > Well, you can generate with the proposed patch and boot with SYSLINUX, > Grub and LILO, but trying to clean Linux real-mode code is where I begun. I haven't looked yet. But I believe this is something that we can do, so long as cleaning and feature enhancement are not mixed in a bad way. >> The big issue is that sometimes bootloaders are a little bit brittle. >> Eric > > I like to think mine is more stable - maybe I am a dreamer... > I do not compete on network booting neither. Phrasing this another way. What we can do with the interface is determined our interface to existing bootloaders and what bootloaders exist. Basically it is the rule that we need to preserve backwards compatibility and changes generally need to be evolutionary ones. Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Eric W. Biederman wrote: > Etienne Lorrain <[EMAIL PROTECTED]> writes: > > New features are more that this real-mode function can return an error > > to the bootloader to tell something to the user, so the user can select > > another kernel with the right processor, another video mode... with > > clean error messages - not a crash dump because this assembly > > instruction is not for that processor. > > Having an error handling compatibility that is backwards compatible sounds > interesting. Well, if the function called at offset 0 in the real-mode section return non zero, that is probably an error - that is Gujin interface, but do not ask me to change other bootloaders to handle that. This function is called with few parameters, one is a string pointer if the function do not want to print the problem report itself. The function is called with an offset of zero in its own intel segment. > > I am still saying that the bootloader knows the root filesystem to > > be used by the kernel it loads, and that ELF is a clean format to > > store different sections to be loaded into memory at predefined > > addresses. > > Yes. Although when you think sections instead of segments I'm a little > worried. Well, for my defense, if Gujin do not find the real mode code in the program header it search for it in the section table, if the program header cannot be modified for some reason. There may be some issues if that ELF file is "stripped" but it seems to work on my side. > > Also there isn't any more kernel size limit. > > I think as HPA points out we have gotten past that a long time > ago with the bzImage format. There was still a test not that long ago (in the code I am not executing) - but I was wrong to bring this point forward. > With the right delicacy, and preserving backwards compatibility > with existing bootloaders I think we can achieve things. Well, you can generate with the proposed patch and boot with SYSLINUX, Grub and LILO, but trying to clean Linux real-mode code is where I begun. > The big issue is that sometimes bootloaders are a little bit brittle. > Eric I like to think mine is more stable - maybe I am a dreamer... I do not compete on network booting neither. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain <[EMAIL PROTECTED]> writes: > H. Peter Anvin wrote: >> Actually, as far as I can see, he has re-invented having a real-mode >> code chunk which then gets run before the protected-mode kernel. We >> already have that! > > I did not claim to have invented anything there, this is just a quite > simple C code to execute instead of the current real mode assembly: > it is a rewrite with obvious advantages/disadvantages. > New features are more that this real-mode function can return an error > to the bootloader to tell something to the user, so the user can select > another kernel with the right processor, another video mode... with > clean error messages - not a crash dump because this assembly > instruction is not for that processor. Having an error handling compatibility that is backwards compatible sounds interesting. > I am still saying that the bootloader knows the root filesystem to > be used by the kernel it loads, and that ELF is a clean format to > store different sections to be loaded into memory at predefined > addresses. Yes. Although when you think sections instead of segments I'm a little worried. > Also there isn't any more kernel size limit. I think as HPA points out we have gotten past that a long time ago with the bzImage format. With the right delicacy, and preserving backwards compatibility with existing bootloaders I think we can achieve things. The big issue is that sometimes bootloaders are a little bit brittle. Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
H. Peter Anvin > Etienne Lorrain wrote: > > Also there isn't any more kernel size limit. > > What kernel size limit? Sorry - there isn't any limit for the kernel size, for a long time. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain wrote: Also there isn't any more kernel size limit. What kernel size limit? -hpa - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
H. Peter Anvin wrote: > Actually, as far as I can see, he has re-invented having a real-mode > code chunk which then gets run before the protected-mode kernel. We > already have that! I did not claim to have invented anything there, this is just a quite simple C code to execute instead of the current real mode assembly: it is a rewrite with obvious advantages/disadvantages. New features are more that this real-mode function can return an error to the bootloader to tell something to the user, so the user can select another kernel with the right processor, another video mode... with clean error messages - not a crash dump because this assembly instruction is not for that processor. I am still saying that the bootloader knows the root filesystem to be used by the kernel it loads, and that ELF is a clean format to store different sections to be loaded into memory at predefined addresses. Also there isn't any more kernel size limit. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Eric W. Biederman wrote: Just for documentation the current interface is you come in through the kernels 16bit entry point and get all of it's BIOS calls, or your bootloader is responsible for getting the equivalent information somehow. From what little I skimmed part of what Gujin wanted to do was sane at first glance. Just boot a gziped vmlinux like the other architectures. The problem was the 16bit code. So there may be some good ideas buried in there somewhere, but it likely to take some doing, and patches that I have to save before I read them are a real pain! Actually, as far as I can see, he has re-invented having a real-mode code chunk which then gets run before the protected-mode kernel. We already have that! You can even intercept between running real-mode and protected-mode if you really want to. I don't really see anything new here. The one thing that he does which we might want to consider emulating is writing the real-mode code in C. Right now, the assembly code is very cluttered, and with the size limit having been, long ago, raised from 2K to 32K we should be able to fit a fair bit of code even if we use C. -hpa - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
H. Peter Anvin, Vivek Goyal wrote: >> Either way, though, putting Gujin-specific code in the main kernel >> output is a pretty dull thud. > > Agreed. May I ask if you are refering to the Gujin structures under the ROOT_EXTENSIVE_SEARCH compilation switch, or the C structures describing the BIOS which could even be used in the kernel? Eric W. Biederman wrote: > From what little I skimmed part of what Gujin wanted to do was sane > at first glance. Just boot a gziped vmlinux like the other > architectures. The problem was the 16bit code. So without ROOT_EXTENSIVE_SEARCH defined. It is so usefull when you have loads of distributions... But yes, maybe that part should be in another file, maybe not a complete directory for a single file. There is also a special problem with ia32, the number of processor involved and their partial compatibilities, that Gujin try to help with. > So there may be some good ideas buried in there somewhere, but it > likely to take some doing, and patches that I have to save before > I read them are a real pain! Last time I used this web interface it did not encode text files... > Eric Thanks, Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
> Building real mode code with kernel binary (vmlinux) has got another > disadvantage that it breaks using vmlinux for kdump purposes. One compiles > the kernel binary to execute from a different address but real mode code/data > will continue to be at virtual/physical addr 0 and kexec can not load it > as that physical memory is not available at all. Kdump skips the real mode > code execution. But that is exactly what you want and need for kdump, isn't it? The ELF file did not change, the program header has the last index at address 0 that you do not want to load because you do not want to execute the real-mode code. Load the rest and provide the 4 Kbytes parameter page - it should work. > I don't know much about Gujin, but advantage here seems to be that it has > capability to load elf files and that's why the attempt to turn kernel binary > into a compressed elf image. Why don't we then simply add an ELF header to > bzImage and Gujin and any ELF loader including Gujin, should be able to load > it? (As Eric had done in one of the implementations in the past?) Why to >create the new infrastructure? Because I think when a program evolves it has to get simpler to generate, run and maintain/debug - while doing more stuff. The number of assembler lines has to reduce because they are difficult to maintain. Removing a ELF header to modify the binary and stick another ELF header is not exactly what I think simpler - a bit like linking at a fix address and then modifying the whole set. HPA wrote: > Well, Gujin wants additional code too. > Putting an ELF header on bzImage broke some bootloaders (GRUB, I believe), > so that's not going to happen again. See the relocatable bzImage thread... I also refuses to load a big file at a fixed address before asking the BIOS for information, I only crash once the memory when I am pretty sure everything seems alright, interruption disabled, just before jumping to the kernel entry point. Cannot do that with bzImage. Cannot easily debug without this feature. > Thanks > Vivek Thanks for you comment. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
> First of all, if sending attached patches, *MAKE SURE* they're text/plain. Sorry, next time I'll do. > The standard way to do this is to have a command line argument named > BOOT_IMAGE (as in BOOT_IMAGE=/foo/bar/baz). There is no reason to do > this differently from every other bootloader. I was more thinking as a command line like the shell command line, I can probably modify that. It is only to guess the root filesystem, I do not know if BOOT_IMAGE= usually contains the fully qualified path. Gujin just adds this string from the "name" field of the GZIP file, so it can be manually adjusted. > Why build the real-mode code as part of the kernel binary? If you have > to reference kernel symbols, you can do that with the -R option to ld. > Bundling it into the kernel binary just to then extract it later is > silly at best. Gujin do not reference any kernel symbol; the complete file is loaded at once at a variable memory address. Then Gujin moves the real-mode section at a valid address and execute it, then it moves the code and data protected mode at a valid address and execute it. In this mode Gujin just act as an ELF program loader. The variable memory address is good when you are running DOS/HIMEM for instance to debug hardware/BIOS - it does not crash random address before requesting BIOS services like all other bootloaders. >> + @echo " /boot/linux-$(KERNELRELEASE).kgz - create a boot kernel >> for the Gujin bootloader" > This doesn't exactly fit very well the standard pattern. Something like > make gujin TARGET=... would be more like it. For me, the make parameter is the file you want to get - up to "make" to sort out how to generate it. > Given how many of your files are highly Gujin-specific, and have generic > names, please put them in a subdirectory (e.g. arch/i386/boot/gujin/). I do agree that there is a functionality I am not sure of the interface, that is the automatic root detection. Nearly everybody put the kernel in its own distribution filesystem (Fedora kernel in Fedora root filesystem, Debian kernel in Debian root filesystem...) or only has a single distribution. The bootloader is loading this kernel file - so it knows exactly which disk/partition to propose as root to the kernel. So under the compilation switch "ROOT_EXTENSIVE_SEARCH", there is the exposed Gujin interface to help the real-mode function of the ELF file decide what to do. I do disagree that the BIOS interface like "union drive_info" of realmode.h is Gujin stuff, it is just that I a lot prefer a C interface with a clean structure than "#define OFFSET_OF_THIS_BYTE, #define MASK_OF_THIS_BIT" everywhere, I prefer programing in C than programing in CPP. Ok, most of the fields are not used - even by Gujin - it is just how the BIOS documentation describes them. >-hpa Thanks for your comments. Sorry I could not answer immediately but I had to sleep a bit. I try to reply to the other messages together. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
First of all, if sending attached patches, *MAKE SURE* they're text/plain. Sorry, next time I'll do. The standard way to do this is to have a command line argument named BOOT_IMAGE (as in BOOT_IMAGE=/foo/bar/baz). There is no reason to do this differently from every other bootloader. I was more thinking as a command line like the shell command line, I can probably modify that. It is only to guess the root filesystem, I do not know if BOOT_IMAGE= usually contains the fully qualified path. Gujin just adds this string from the name field of the GZIP file, so it can be manually adjusted. Why build the real-mode code as part of the kernel binary? If you have to reference kernel symbols, you can do that with the -R option to ld. Bundling it into the kernel binary just to then extract it later is silly at best. Gujin do not reference any kernel symbol; the complete file is loaded at once at a variable memory address. Then Gujin moves the real-mode section at a valid address and execute it, then it moves the code and data protected mode at a valid address and execute it. In this mode Gujin just act as an ELF program loader. The variable memory address is good when you are running DOS/HIMEM for instance to debug hardware/BIOS - it does not crash random address before requesting BIOS services like all other bootloaders. + @echo/boot/linux-$(KERNELRELEASE).kgz - create a boot kernel for the Gujin bootloader This doesn't exactly fit very well the standard pattern. Something like make gujin TARGET=... would be more like it. For me, the make parameter is the file you want to get - up to make to sort out how to generate it. Given how many of your files are highly Gujin-specific, and have generic names, please put them in a subdirectory (e.g. arch/i386/boot/gujin/). I do agree that there is a functionality I am not sure of the interface, that is the automatic root detection. Nearly everybody put the kernel in its own distribution filesystem (Fedora kernel in Fedora root filesystem, Debian kernel in Debian root filesystem...) or only has a single distribution. The bootloader is loading this kernel file - so it knows exactly which disk/partition to propose as root to the kernel. So under the compilation switch ROOT_EXTENSIVE_SEARCH, there is the exposed Gujin interface to help the real-mode function of the ELF file decide what to do. I do disagree that the BIOS interface like union drive_info of realmode.h is Gujin stuff, it is just that I a lot prefer a C interface with a clean structure than #define OFFSET_OF_THIS_BYTE, #define MASK_OF_THIS_BIT everywhere, I prefer programing in C than programing in CPP. Ok, most of the fields are not used - even by Gujin - it is just how the BIOS documentation describes them. -hpa Thanks for your comments. Sorry I could not answer immediately but I had to sleep a bit. I try to reply to the other messages together. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Building real mode code with kernel binary (vmlinux) has got another disadvantage that it breaks using vmlinux for kdump purposes. One compiles the kernel binary to execute from a different address but real mode code/data will continue to be at virtual/physical addr 0 and kexec can not load it as that physical memory is not available at all. Kdump skips the real mode code execution. But that is exactly what you want and need for kdump, isn't it? The ELF file did not change, the program header has the last index at address 0 that you do not want to load because you do not want to execute the real-mode code. Load the rest and provide the 4 Kbytes parameter page - it should work. I don't know much about Gujin, but advantage here seems to be that it has capability to load elf files and that's why the attempt to turn kernel binary into a compressed elf image. Why don't we then simply add an ELF header to bzImage and Gujin and any ELF loader including Gujin, should be able to load it? (As Eric had done in one of the implementations in the past?) Why to create the new infrastructure? Because I think when a program evolves it has to get simpler to generate, run and maintain/debug - while doing more stuff. The number of assembler lines has to reduce because they are difficult to maintain. Removing a ELF header to modify the binary and stick another ELF header is not exactly what I think simpler - a bit like linking at a fix address and then modifying the whole set. HPA wrote: Well, Gujin wants additional code too. Putting an ELF header on bzImage broke some bootloaders (GRUB, I believe), so that's not going to happen again. See the relocatable bzImage thread... I also refuses to load a big file at a fixed address before asking the BIOS for information, I only crash once the memory when I am pretty sure everything seems alright, interruption disabled, just before jumping to the kernel entry point. Cannot do that with bzImage. Cannot easily debug without this feature. Thanks Vivek Thanks for you comment. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
H. Peter Anvin, Vivek Goyal wrote: Either way, though, putting Gujin-specific code in the main kernel output is a pretty dull thud. Agreed. May I ask if you are refering to the Gujin structures under the ROOT_EXTENSIVE_SEARCH compilation switch, or the C structures describing the BIOS which could even be used in the kernel? Eric W. Biederman wrote: From what little I skimmed part of what Gujin wanted to do was sane at first glance. Just boot a gziped vmlinux like the other architectures. The problem was the 16bit code. So without ROOT_EXTENSIVE_SEARCH defined. It is so usefull when you have loads of distributions... But yes, maybe that part should be in another file, maybe not a complete directory for a single file. There is also a special problem with ia32, the number of processor involved and their partial compatibilities, that Gujin try to help with. So there may be some good ideas buried in there somewhere, but it likely to take some doing, and patches that I have to save before I read them are a real pain! Last time I used this web interface it did not encode text files... Eric Thanks, Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Eric W. Biederman wrote: Just for documentation the current interface is you come in through the kernels 16bit entry point and get all of it's BIOS calls, or your bootloader is responsible for getting the equivalent information somehow. From what little I skimmed part of what Gujin wanted to do was sane at first glance. Just boot a gziped vmlinux like the other architectures. The problem was the 16bit code. So there may be some good ideas buried in there somewhere, but it likely to take some doing, and patches that I have to save before I read them are a real pain! Actually, as far as I can see, he has re-invented having a real-mode code chunk which then gets run before the protected-mode kernel. We already have that! You can even intercept between running real-mode and protected-mode if you really want to. I don't really see anything new here. The one thing that he does which we might want to consider emulating is writing the real-mode code in C. Right now, the assembly code is very cluttered, and with the size limit having been, long ago, raised from 2K to 32K we should be able to fit a fair bit of code even if we use C. -hpa - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
H. Peter Anvin wrote: Actually, as far as I can see, he has re-invented having a real-mode code chunk which then gets run before the protected-mode kernel. We already have that! I did not claim to have invented anything there, this is just a quite simple C code to execute instead of the current real mode assembly: it is a rewrite with obvious advantages/disadvantages. New features are more that this real-mode function can return an error to the bootloader to tell something to the user, so the user can select another kernel with the right processor, another video mode... with clean error messages - not a crash dump because this assembly instruction is not for that processor. I am still saying that the bootloader knows the root filesystem to be used by the kernel it loads, and that ELF is a clean format to store different sections to be loaded into memory at predefined addresses. Also there isn't any more kernel size limit. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain wrote: Also there isn't any more kernel size limit. What kernel size limit? -hpa - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
H. Peter Anvin Etienne Lorrain wrote: Also there isn't any more kernel size limit. What kernel size limit? Sorry - there isn't any limit for the kernel size, for a long time. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain [EMAIL PROTECTED] writes: H. Peter Anvin wrote: Actually, as far as I can see, he has re-invented having a real-mode code chunk which then gets run before the protected-mode kernel. We already have that! I did not claim to have invented anything there, this is just a quite simple C code to execute instead of the current real mode assembly: it is a rewrite with obvious advantages/disadvantages. New features are more that this real-mode function can return an error to the bootloader to tell something to the user, so the user can select another kernel with the right processor, another video mode... with clean error messages - not a crash dump because this assembly instruction is not for that processor. Having an error handling compatibility that is backwards compatible sounds interesting. I am still saying that the bootloader knows the root filesystem to be used by the kernel it loads, and that ELF is a clean format to store different sections to be loaded into memory at predefined addresses. Yes. Although when you think sections instead of segments I'm a little worried. Also there isn't any more kernel size limit. I think as HPA points out we have gotten past that a long time ago with the bzImage format. With the right delicacy, and preserving backwards compatibility with existing bootloaders I think we can achieve things. The big issue is that sometimes bootloaders are a little bit brittle. Eric - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Eric W. Biederman wrote: Etienne Lorrain [EMAIL PROTECTED] writes: New features are more that this real-mode function can return an error to the bootloader to tell something to the user, so the user can select another kernel with the right processor, another video mode... with clean error messages - not a crash dump because this assembly instruction is not for that processor. Having an error handling compatibility that is backwards compatible sounds interesting. Well, if the function called at offset 0 in the real-mode section return non zero, that is probably an error - that is Gujin interface, but do not ask me to change other bootloaders to handle that. This function is called with few parameters, one is a string pointer if the function do not want to print the problem report itself. The function is called with an offset of zero in its own intel segment. I am still saying that the bootloader knows the root filesystem to be used by the kernel it loads, and that ELF is a clean format to store different sections to be loaded into memory at predefined addresses. Yes. Although when you think sections instead of segments I'm a little worried. Well, for my defense, if Gujin do not find the real mode code in the program header it search for it in the section table, if the program header cannot be modified for some reason. There may be some issues if that ELF file is stripped but it seems to work on my side. Also there isn't any more kernel size limit. I think as HPA points out we have gotten past that a long time ago with the bzImage format. There was still a test not that long ago (in the code I am not executing) - but I was wrong to bring this point forward. With the right delicacy, and preserving backwards compatibility with existing bootloaders I think we can achieve things. Well, you can generate with the proposed patch and boot with SYSLINUX, Grub and LILO, but trying to clean Linux real-mode code is where I begun. The big issue is that sometimes bootloaders are a little bit brittle. Eric I like to think mine is more stable - maybe I am a dreamer... I do not compete on network booting neither. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain [EMAIL PROTECTED] writes: Eric W. Biederman wrote: Etienne Lorrain [EMAIL PROTECTED] writes: New features are more that this real-mode function can return an error to the bootloader to tell something to the user, so the user can select another kernel with the right processor, another video mode... with clean error messages - not a crash dump because this assembly instruction is not for that processor. Having an error handling compatibility that is backwards compatible sounds interesting. Well, if the function called at offset 0 in the real-mode section return non zero, that is probably an error - that is Gujin interface, but do not ask me to change other bootloaders to handle that. This function is called with few parameters, one is a string pointer if the function do not want to print the problem report itself. The function is called with an offset of zero in its own intel segment. So I think there are some backwards compatibility issues with your real mode code interface. Setting a new flag that says to return instead of printing an error message and halting would be one way to handle this. Is your real mode C code section relative such that it can be loaded at different addresses and still work? I am still saying that the bootloader knows the root filesystem to be used by the kernel it loads, and that ELF is a clean format to store different sections to be loaded into memory at predefined addresses. Yes. Although when you think sections instead of segments I'm a little worried. Well, for my defense, if Gujin do not find the real mode code in the program header it search for it in the section table, if the program header cannot be modified for some reason. There may be some issues if that ELF file is stripped but it seems to work on my side. The program header is for executable loaders the section header is for linkers and the section header is optional in PT_LOAD and PT_DYN executables. Use of the section header by a loader is a bug. Also there isn't any more kernel size limit. I think as HPA points out we have gotten past that a long time ago with the bzImage format. There was still a test not that long ago (in the code I am not executing) - but I was wrong to bring this point forward. There have been limitations but mostly with respect to page table size and the like but they were not limitations a bootloader had to deal with. With the right delicacy, and preserving backwards compatibility with existing bootloaders I think we can achieve things. Well, you can generate with the proposed patch and boot with SYSLINUX, Grub and LILO, but trying to clean Linux real-mode code is where I begun. I haven't looked yet. But I believe this is something that we can do, so long as cleaning and feature enhancement are not mixed in a bad way. The big issue is that sometimes bootloaders are a little bit brittle. Eric I like to think mine is more stable - maybe I am a dreamer... I do not compete on network booting neither. Phrasing this another way. What we can do with the interface is determined our interface to existing bootloaders and what bootloaders exist. Basically it is the rule that we need to preserve backwards compatibility and changes generally need to be evolutionary ones. Eric - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Eric W. Biederman wrote: Etienne Lorrain [EMAIL PROTECTED] writes: Well, if the function called at offset 0 in the real-mode section return non zero, that is probably an error - that is Gujin interface, but do not ask me to change other bootloaders to handle that. This function is called with few parameters, one is a string pointer if the function do not want to print the problem report itself. The function is called with an offset of zero in its own intel segment. So I think there are some backwards compatibility issues with your real mode code interface. Setting a new flag that says to return instead of printing an error message and halting would be one way to handle this. I am not sure to understand: Gujin calls a function inside the ELF real mode program header of the Linux kernel. All the system is currently in real mode. There isn't any limitation in this function of the kernel to decide to print some message and not return at all, this function can selectively do so by reading which bootloader ID it is using (another parameter). It is not done mostly because printf is difficult to do in assembler, Gujin has no problem. Is your real mode C code section relative such that it can be loaded at different addresses and still work? The code is relative - but not the data (first 4 Kbytes at %ds:0 and stack available) - but the whole lot at any segment boundary, i.e. every 16 bytes. Else Gujin would not work under DOS. The program header is for executable loaders the section header is for linkers and the section header is optional in PT_LOAD and PT_DYN executables. Use of the section header by a loader is a bug. Unless if there is a problem, Gujin uses only the program header; it has a look in the section header just in case - that can be removed easily. I was wondering about a relocatable image there - but I do not know enough for that. There have been limitations but mostly with respect to page table size and the like but they were not limitations a bootloader had to deal with. Was talking about this comment, but it is old: /* 0x28000*16 = 2.5 MB, conservative estimate for the current maximum */ http://lxr.linux.no/source/arch/i386/boot/tools/build.c?v=2.4.28 Well, you can generate with the proposed patch and boot with SYSLINUX, Grub and LILO, but trying to clean Linux real-mode code is where I begun. I haven't looked yet. But I believe this is something that we can do, so long as cleaning and feature enhancement are not mixed in a bad way. Phrasing this another way. What we can do with the interface is determined our interface to existing bootloaders and what bootloaders exist. Basically it is the rule that we need to preserve backwards compatibility and changes generally need to be evolutionary ones. Well, if you want to preserve compatibility with other bootloaders, it is probably possible to put some source around the ELF kernel file, mostly taken from Gujin if you want (GPL), but I wonder why you would like to be compatible with LILO. Modifying the linux real mode assembler, nobody could even want to. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain [EMAIL PROTECTED] writes: I am not sure to understand: Gujin calls a function inside the ELF real mode program header of the Linux kernel. All the system is currently in real mode. There isn't any limitation in this function of the kernel to decide to print some message and not return at all, this function can selectively do so by reading which bootloader ID it is using (another parameter). It is not done mostly because printf is difficult to do in assembler, Gujin has no problem. This code is currently completely Gujin specific. If concieved as a replacement for setup.S it has a chance of passing review. Is your real mode C code section relative such that it can be loaded at different addresses and still work? The code is relative - but not the data (first 4 Kbytes at %ds:0 and stack available) - but the whole lot at any segment boundary, i.e. every 16 bytes. Else Gujin would not work under DOS. Ok. Similar restrictions to the current real mode code. The program header is for executable loaders the section header is for linkers and the section header is optional in PT_LOAD and PT_DYN executables. Use of the section header by a loader is a bug. Unless if there is a problem, Gujin uses only the program header; it has a look in the section header just in case - that can be removed easily. I was wondering about a relocatable image there - but I do not know enough for that. When you are relocatable no relocations are processed and all addresses in the program header are slid by some fixed amount (preserving alignment) after that the relocated object has to work out the relocations by itself. Well, you can generate with the proposed patch and boot with SYSLINUX, Grub and LILO, but trying to clean Linux real-mode code is where I begun. I haven't looked yet. But I believe this is something that we can do, so long as cleaning and feature enhancement are not mixed in a bad way. Phrasing this another way. What we can do with the interface is determined our interface to existing bootloaders and what bootloaders exist. Basically it is the rule that we need to preserve backwards compatibility and changes generally need to be evolutionary ones. Well, if you want to preserve compatibility with other bootloaders, it is probably possible to put some source around the ELF kernel file, mostly taken from Gujin if you want (GPL), but I wonder why you would like to be compatible with LILO. LILO is a lot saner then Grub, and it still supports more filesystems... Just because it memorizes it all before you shut down the system for simplicity doesn't mean lilo is worse. Modifying the linux real mode assembler, nobody could even want to. I have several times. It's just code. But I do agree it will likely be more maintainable if we can convert it to C. On that same token I wrote a compiler in romcc in another context so I didn't have to write so much assembly. And for maintainability and comprehensibility it was a big help. Eric - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Eric W. Biederman wrote: Modifying the linux real mode assembler, nobody could even want to. I have several times. It's just code. But I do agree it will likely be more maintainable if we can convert it to C. On that same token I wrote a compiler in romcc in another context so I didn't have to write so much assembly. And for maintainability and comprehensibility it was a big help. I've long wished that someone would do a proper 16-bit x86 port of gcc; however, the .code16gcc is usually good enough, although it produces code which is a lot bigger than it needs to be. -hpa - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re : Re : Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
H. Peter Anvin wrote: I've long wished that someone would do a proper 16-bit x86 port of gcc; however, the .code16gcc is usually good enough, although it produces code which is a lot bigger than it needs to be. It is only that much bigger if you compare to 16 bits integer compilers, but once you deal with 32 bits integers in real mode you have one extra byte out of the two - which is still better than dealing with dx:ax. Last time I compiled (long time ago) some piece of code with and without .code16gcc it was approx 15% bigger - approx what you get when optimising. Etienne. ___ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : Re : Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Etienne Lorrain wrote: H. Peter Anvin wrote: I've long wished that someone would do a proper 16-bit x86 port of gcc; however, the .code16gcc is usually good enough, although it produces code which is a lot bigger than it needs to be. It is only that much bigger if you compare to 16 bits integer compilers, but once you deal with 32 bits integers in real mode you have one extra byte out of the two - which is still better than dealing with dx:ax. Last time I compiled (long time ago) some piece of code with and without .code16gcc it was approx 15% bigger - approx what you get when optimising. That's not a valid comparison, because you're using 32-bit registers to hold 16-bit pointers, and then sticking 67 prefixes on completely unnecessarily. -hpa - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : Re : Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
H. Peter Anvin [EMAIL PROTECTED] writes: Etienne Lorrain wrote: H. Peter Anvin wrote: I've long wished that someone would do a proper 16-bit x86 port of gcc; however, the .code16gcc is usually good enough, although it produces code which is a lot bigger than it needs to be. It is only that much bigger if you compare to 16 bits integer compilers, but once you deal with 32 bits integers in real mode you have one extra byte out of the two - which is still better than dealing with dx:ax. Last time I compiled (long time ago) some piece of code with and without .code16gcc it was approx 15% bigger - approx what you get when optimising. That's not a valid comparison, because you're using 32-bit registers to hold 16-bit pointers, and then sticking 67 prefixes on completely unnecessarily. Regardless if the size is good enough we can use it :) For romcc I had a 3x code bloat over had crafted assembler, because I had no memory and had to inline every single function call because I didn't have someplace to put a return register. So I have the scenario with the most code bloat. I win! Eric - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
On Tue, Feb 06, 2007 at 01:12:05PM +, Etienne Lorrain wrote: Building real mode code with kernel binary (vmlinux) has got another disadvantage that it breaks using vmlinux for kdump purposes. One compiles the kernel binary to execute from a different address but real mode code/data will continue to be at virtual/physical addr 0 and kexec can not load it as that physical memory is not available at all. Kdump skips the real mode code execution. But that is exactly what you want and need for kdump, isn't it? The ELF file did not change, the program header has the last index at address 0 that you do not want to load because you do not want to execute the real-mode code. Load the rest and provide the 4 Kbytes parameter page - it should work. How do I know which program header is real mode code and the boot loader is not supposed to load it? May be PT_LOAD header with physical addr 0? What happens if changes happen and down the line we start compiling real mode code for non-zero address? Hence I think keeping real mode code out of vmlinux might prove to be a good idea. Secondly, if you compile real mode code with vmlinux, what would be the entry point for this ELF file? Real mode entry? Then I have not way to find out from ELF headers where is the protected mode entry point and I can not do use this vmlinux with kexec/kdump. OTOH, now bzImage is relocatable. Is this image going to be relocatable? How do we take care of that? Thanks Vivek - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Vivek Goyal <[EMAIL PROTECTED]> writes: > On Mon, Feb 05, 2007 at 10:38:30PM -0800, H. Peter Anvin wrote: >> Vivek Goyal wrote: >> > >> >That time, grub was working fine but lilo had broken on one of Andrew's >> >machine. I could not reproduce the problem on my setup and had no clue what >> >was the issue. Hence got rid of ELF header from bzImage which was >> >non-essential >> >stuff for relocatable kernel. But that issue should be fixable. >> > >> >> Either way, though, putting Gujin-specific code in the main kernel >> output is a pretty dull thud. >> > > Agreed. Just for documentation the current interface is you come in through the kernels 16bit entry point and get all of it's BIOS calls, or your bootloader is responsible for getting the equivalent information somehow. >From what little I skimmed part of what Gujin wanted to do was sane at first glance. Just boot a gziped vmlinux like the other architectures. The problem was the 16bit code. So there may be some good ideas buried in there somewhere, but it likely to take some doing, and patches that I have to save before I read them are a real pain! Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
On Mon, Feb 05, 2007 at 10:38:30PM -0800, H. Peter Anvin wrote: > Vivek Goyal wrote: > > > >That time, grub was working fine but lilo had broken on one of Andrew's > >machine. I could not reproduce the problem on my setup and had no clue what > >was the issue. Hence got rid of ELF header from bzImage which was > >non-essential > >stuff for relocatable kernel. But that issue should be fixable. > > > > Either way, though, putting Gujin-specific code in the main kernel > output is a pretty dull thud. > Agreed. Thanks Vivek - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Vivek Goyal wrote: That time, grub was working fine but lilo had broken on one of Andrew's machine. I could not reproduce the problem on my setup and had no clue what was the issue. Hence got rid of ELF header from bzImage which was non-essential stuff for relocatable kernel. But that issue should be fixable. Either way, though, putting Gujin-specific code in the main kernel output is a pretty dull thud. -hpa - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
On Mon, Feb 05, 2007 at 10:16:59PM -0800, H. Peter Anvin wrote: > Vivek Goyal wrote: > > > >I don't know much about Gujin, but advantage here seems to be that it has > >capability to load elf files and that's why the attempt to turn kernel > >binary > >into a compressed elf image. Why don't we then simply add an ELF header to > >bzImage and Gujin and any ELF loader including Gujin, should be able to > >load > >it? (As Eric had done in one of the implementations in the past?) Why to > >create the new infrastructure? > > > > Well, Gujin wants additional code too. > > Putting an ELF header on bzImage broke some bootloaders (GRUB, I > believe), so that's not going to happen again. See the relocatable > bzImage thread... > That time, grub was working fine but lilo had broken on one of Andrew's machine. I could not reproduce the problem on my setup and had no clue what was the issue. Hence got rid of ELF header from bzImage which was non-essential stuff for relocatable kernel. But that issue should be fixable. Thanks Vivek - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Vivek Goyal wrote: I don't know much about Gujin, but advantage here seems to be that it has capability to load elf files and that's why the attempt to turn kernel binary into a compressed elf image. Why don't we then simply add an ELF header to bzImage and Gujin and any ELF loader including Gujin, should be able to load it? (As Eric had done in one of the implementations in the past?) Why to create the new infrastructure? Well, Gujin wants additional code too. Putting an ELF header on bzImage broke some bootloaders (GRUB, I believe), so that's not going to happen again. See the relocatable bzImage thread... -hpa - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
On Mon, Feb 05, 2007 at 04:37:43PM -0800, H. Peter Anvin wrote: > First of all, if sending attached patches, *MAKE SURE* they're text/plain. > > >diff -uprN -X linux-2.6.20/Documentation/dontdiff > >linux-2.6.20/arch/i386/kernel/setup.c > >linux-2.6.20-gujin/arch/i386/kernel/setup.c > >--- linux-2.6.20/arch/i386/kernel/setup.c 2007-02-04 > >18:44:54.0 + > >+++ linux-2.6.20-gujin/arch/i386/kernel/setup.c 2007-02-05 > >21:27:01.0 + > >@@ -579,6 +579,19 @@ void __init setup_arch(char **cmdline_p) > >strlcpy(command_line, saved_command_line, COMMAND_LINE_SIZE); > >*cmdline_p = command_line; > > > >+ /* > >+* If the command line begin with '/', it is the filename of the > >+* kernel file - like a shell command line. That can be used to > >+* deduce where root was located when the kernel was compiled, > >+* inside a directory of root named /boot, inside a small partition > >+* mounted on a directory of root named /boot or in the root > >+* directory itself. > >+*/ > >+ if (**cmdline_p == '/') > >+ while (**cmdline_p != ' ' && **cmdline_p != '\t' > >+ && **cmdline_p != '\n' && **cmdline_p != '\0') > >+ ++*cmdline_p; > >+ > >max_low_pfn = setup_memory(); > > > >/* > > The standard way to do this is to have a command line argument named > BOOT_IMAGE (as in BOOT_IMAGE=/foo/bar/baz). There is no reason to do > this differently from every other bootloader. > > Why build the real-mode code as part of the kernel binary? If you have > to reference kernel symbols, you can do that with the -R option to ld. > Bundling it into the kernel binary just to then extract it later is > silly at best. > Building real mode code with kernel binary (vmlinux) has got another disadvantage that it breaks using vmlinux for kdump purposes. One compiles the kernel binary to execute from a different address but real mode code/data will continue to be at virtual/physical addr 0 and kexec can not load it as that physical memory is not available at all. Kdump skips the real mode code execution. I don't know much about Gujin, but advantage here seems to be that it has capability to load elf files and that's why the attempt to turn kernel binary into a compressed elf image. Why don't we then simply add an ELF header to bzImage and Gujin and any ELF loader including Gujin, should be able to load it? (As Eric had done in one of the implementations in the past?) Why to create the new infrastructure? Thanks Vivek - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
First of all, if sending attached patches, *MAKE SURE* they're text/plain. diff -uprN -X linux-2.6.20/Documentation/dontdiff linux-2.6.20/arch/i386/kernel/setup.c linux-2.6.20-gujin/arch/i386/kernel/setup.c --- linux-2.6.20/arch/i386/kernel/setup.c 2007-02-04 18:44:54.0 + +++ linux-2.6.20-gujin/arch/i386/kernel/setup.c 2007-02-05 21:27:01.0 + @@ -579,6 +579,19 @@ void __init setup_arch(char **cmdline_p) strlcpy(command_line, saved_command_line, COMMAND_LINE_SIZE); *cmdline_p = command_line; + /* +* If the command line begin with '/', it is the filename of the +* kernel file - like a shell command line. That can be used to +* deduce where root was located when the kernel was compiled, +* inside a directory of root named /boot, inside a small partition +* mounted on a directory of root named /boot or in the root +* directory itself. +*/ + if (**cmdline_p == '/') + while (**cmdline_p != ' ' && **cmdline_p != '\t' + && **cmdline_p != '\n' && **cmdline_p != '\0') + ++*cmdline_p; + max_low_pfn = setup_memory(); /* The standard way to do this is to have a command line argument named BOOT_IMAGE (as in BOOT_IMAGE=/foo/bar/baz). There is no reason to do this differently from every other bootloader. Why build the real-mode code as part of the kernel binary? If you have to reference kernel symbols, you can do that with the -R option to ld. Bundling it into the kernel binary just to then extract it later is silly at best. @@ -1106,6 +1179,9 @@ help: @echo 'Other generic targets:' @echo ' all - Build all targets marked with [*]' @echo '* vmlinux - Build the bare kernel' + @echo " /boot/linux-$(KERNELRELEASE).kgz - create a boot kernel for the Gujin bootloader" + @echo ' /boot/linux.kgz ROOT=/dev/hda1 - do not autodetect root filesystem at boot time' + @echo ' /boot/linux.kgz ROOT=auto- root filesystem from current /proc/cmdline' @echo '* modules - Build all modules' @echo ' modules_install - Install all modules to INSTALL_MOD_PATH (default: /)' @echo ' dir/- Build all files in dir and below' This doesn't exactly fit very well the standard pattern. Something like make gujin TARGET=... would be more like it. Given how many of your files are highly Gujin-specific, and have generic names, please put them in a subdirectory (e.g. arch/i386/boot/gujin/). -hpa - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
First of all, if sending attached patches, *MAKE SURE* they're text/plain. diff -uprN -X linux-2.6.20/Documentation/dontdiff linux-2.6.20/arch/i386/kernel/setup.c linux-2.6.20-gujin/arch/i386/kernel/setup.c --- linux-2.6.20/arch/i386/kernel/setup.c 2007-02-04 18:44:54.0 + +++ linux-2.6.20-gujin/arch/i386/kernel/setup.c 2007-02-05 21:27:01.0 + @@ -579,6 +579,19 @@ void __init setup_arch(char **cmdline_p) strlcpy(command_line, saved_command_line, COMMAND_LINE_SIZE); *cmdline_p = command_line; + /* +* If the command line begin with '/', it is the filename of the +* kernel file - like a shell command line. That can be used to +* deduce where root was located when the kernel was compiled, +* inside a directory of root named /boot, inside a small partition +* mounted on a directory of root named /boot or in the root +* directory itself. +*/ + if (**cmdline_p == '/') + while (**cmdline_p != ' ' **cmdline_p != '\t' +**cmdline_p != '\n' **cmdline_p != '\0') + ++*cmdline_p; + max_low_pfn = setup_memory(); /* The standard way to do this is to have a command line argument named BOOT_IMAGE (as in BOOT_IMAGE=/foo/bar/baz). There is no reason to do this differently from every other bootloader. Why build the real-mode code as part of the kernel binary? If you have to reference kernel symbols, you can do that with the -R option to ld. Bundling it into the kernel binary just to then extract it later is silly at best. @@ -1106,6 +1179,9 @@ help: @echo 'Other generic targets:' @echo ' all - Build all targets marked with [*]' @echo '* vmlinux - Build the bare kernel' + @echo/boot/linux-$(KERNELRELEASE).kgz - create a boot kernel for the Gujin bootloader + @echo ' /boot/linux.kgz ROOT=/dev/hda1 - do not autodetect root filesystem at boot time' + @echo ' /boot/linux.kgz ROOT=auto- root filesystem from current /proc/cmdline' @echo '* modules - Build all modules' @echo ' modules_install - Install all modules to INSTALL_MOD_PATH (default: /)' @echo ' dir/- Build all files in dir and below' This doesn't exactly fit very well the standard pattern. Something like make gujin TARGET=... would be more like it. Given how many of your files are highly Gujin-specific, and have generic names, please put them in a subdirectory (e.g. arch/i386/boot/gujin/). -hpa - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
On Mon, Feb 05, 2007 at 04:37:43PM -0800, H. Peter Anvin wrote: First of all, if sending attached patches, *MAKE SURE* they're text/plain. diff -uprN -X linux-2.6.20/Documentation/dontdiff linux-2.6.20/arch/i386/kernel/setup.c linux-2.6.20-gujin/arch/i386/kernel/setup.c --- linux-2.6.20/arch/i386/kernel/setup.c 2007-02-04 18:44:54.0 + +++ linux-2.6.20-gujin/arch/i386/kernel/setup.c 2007-02-05 21:27:01.0 + @@ -579,6 +579,19 @@ void __init setup_arch(char **cmdline_p) strlcpy(command_line, saved_command_line, COMMAND_LINE_SIZE); *cmdline_p = command_line; + /* +* If the command line begin with '/', it is the filename of the +* kernel file - like a shell command line. That can be used to +* deduce where root was located when the kernel was compiled, +* inside a directory of root named /boot, inside a small partition +* mounted on a directory of root named /boot or in the root +* directory itself. +*/ + if (**cmdline_p == '/') + while (**cmdline_p != ' ' **cmdline_p != '\t' +**cmdline_p != '\n' **cmdline_p != '\0') + ++*cmdline_p; + max_low_pfn = setup_memory(); /* The standard way to do this is to have a command line argument named BOOT_IMAGE (as in BOOT_IMAGE=/foo/bar/baz). There is no reason to do this differently from every other bootloader. Why build the real-mode code as part of the kernel binary? If you have to reference kernel symbols, you can do that with the -R option to ld. Bundling it into the kernel binary just to then extract it later is silly at best. Building real mode code with kernel binary (vmlinux) has got another disadvantage that it breaks using vmlinux for kdump purposes. One compiles the kernel binary to execute from a different address but real mode code/data will continue to be at virtual/physical addr 0 and kexec can not load it as that physical memory is not available at all. Kdump skips the real mode code execution. I don't know much about Gujin, but advantage here seems to be that it has capability to load elf files and that's why the attempt to turn kernel binary into a compressed elf image. Why don't we then simply add an ELF header to bzImage and Gujin and any ELF loader including Gujin, should be able to load it? (As Eric had done in one of the implementations in the past?) Why to create the new infrastructure? Thanks Vivek - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Vivek Goyal wrote: I don't know much about Gujin, but advantage here seems to be that it has capability to load elf files and that's why the attempt to turn kernel binary into a compressed elf image. Why don't we then simply add an ELF header to bzImage and Gujin and any ELF loader including Gujin, should be able to load it? (As Eric had done in one of the implementations in the past?) Why to create the new infrastructure? Well, Gujin wants additional code too. Putting an ELF header on bzImage broke some bootloaders (GRUB, I believe), so that's not going to happen again. See the relocatable bzImage thread... -hpa - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
On Mon, Feb 05, 2007 at 10:16:59PM -0800, H. Peter Anvin wrote: Vivek Goyal wrote: I don't know much about Gujin, but advantage here seems to be that it has capability to load elf files and that's why the attempt to turn kernel binary into a compressed elf image. Why don't we then simply add an ELF header to bzImage and Gujin and any ELF loader including Gujin, should be able to load it? (As Eric had done in one of the implementations in the past?) Why to create the new infrastructure? Well, Gujin wants additional code too. Putting an ELF header on bzImage broke some bootloaders (GRUB, I believe), so that's not going to happen again. See the relocatable bzImage thread... That time, grub was working fine but lilo had broken on one of Andrew's machine. I could not reproduce the problem on my setup and had no clue what was the issue. Hence got rid of ELF header from bzImage which was non-essential stuff for relocatable kernel. But that issue should be fixable. Thanks Vivek - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Vivek Goyal wrote: That time, grub was working fine but lilo had broken on one of Andrew's machine. I could not reproduce the problem on my setup and had no clue what was the issue. Hence got rid of ELF header from bzImage which was non-essential stuff for relocatable kernel. But that issue should be fixable. Either way, though, putting Gujin-specific code in the main kernel output is a pretty dull thud. -hpa - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
On Mon, Feb 05, 2007 at 10:38:30PM -0800, H. Peter Anvin wrote: Vivek Goyal wrote: That time, grub was working fine but lilo had broken on one of Andrew's machine. I could not reproduce the problem on my setup and had no clue what was the issue. Hence got rid of ELF header from bzImage which was non-essential stuff for relocatable kernel. But that issue should be fixable. Either way, though, putting Gujin-specific code in the main kernel output is a pretty dull thud. Agreed. Thanks Vivek - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
Vivek Goyal [EMAIL PROTECTED] writes: On Mon, Feb 05, 2007 at 10:38:30PM -0800, H. Peter Anvin wrote: Vivek Goyal wrote: That time, grub was working fine but lilo had broken on one of Andrew's machine. I could not reproduce the problem on my setup and had no clue what was the issue. Hence got rid of ELF header from bzImage which was non-essential stuff for relocatable kernel. But that issue should be fixable. Either way, though, putting Gujin-specific code in the main kernel output is a pretty dull thud. Agreed. Just for documentation the current interface is you come in through the kernels 16bit entry point and get all of it's BIOS calls, or your bootloader is responsible for getting the equivalent information somehow. From what little I skimmed part of what Gujin wanted to do was sane at first glance. Just boot a gziped vmlinux like the other architectures. The problem was the 16bit code. So there may be some good ideas buried in there somewhere, but it likely to take some doing, and patches that I have to save before I read them are a real pain! Eric - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
