Re: [coreboot] About Paging, Realmode and what is going on

[ron minnich]

On Thu, Sep 7, 2017 at 1:43 AM Zoran Stojsavljevic <
zoran.stojsavlje...@gmail.com> wrote:

>
> Please, could you try to do this with INTEL ATOM or CORE in this time for
> this price? ;-)
>
>
>
This is a very interesting point about time to market, which is everything
nowadays. I regret that ARM decided to take such a strong UEFI position
with the v8 but maybe the market will force them back to more openness.
Zoran, you make a good economic point for openness.

Let's hope RISCV avoids the same UEFI mistake, though companies like HP
seem to want to push them into UEFI.
-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Peter Stuge]

Julius Werner wrote:
> while the program was running it mostly interacted with the BIOS directly.

Not in general - that is/was highly dependent on the program.

BIOS interrupt services offer a hardware abstraction, but they come
at a (high! interrupts are very expensive!) cost.

For anything performance sensitive, e.g. games, multitasking environments
(DESQview, Windows) or terminal emulators communicating via serial ports
which may or may not have a FIFO, the BIOS overhead is simply unacceptable.

Such programs instead interacted directly with the hardware, having
implemented drivers for some graphics and sound cards, UARTs etc.

Even the mouse BIOS interrupt services are pretty much unusable for
anything serious.


//Peter

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Zoran Stojsavljevic]

> Linux, for a long time, did not need no steenkin' BIOS (you can find that
quote if you look back far enough).

*But those days are gone.*
This is the reason why INTEL has no chance to win IoT race with ARM (just
in few special use cases). With all these compatibilities, and test modes,
INTEL is forcing customers to the HW CLOSED model, forcing as well Firmware
CLOSED model (with 10x more silicon/HW extensions to comply, and end
users/customers are paying for that). Since there are gazillion of boot
registers which need to be set-up properly for modes to work as intended.

Having concept of Device Tree Usage (HW OPEN model, forcing FW/BSP OPEN
model), I can, having one very smart HW designer (with random IO controller
order with ARM SoCs: i.MX6/7/8, tegra, etc.), build the whole ARM based SoC
HW and whole system SW including FW, application tools rich root-tree, and
Device Drivers in few weeks using YOCTO. Board Plug & Play, ready for
Application, written in Qt5, Java, C++ with all tools, U name it!

https://events.linuxfoundation.org/sites/events/files/slides/petazzoni-
device-tree-dummies.pdf

So, Device Tree Usage + YOCTO = 2 experienced men work from scratch with
the COMPLETE embedded solution for 2 to 3 weeks. Mini com express for
around 20 to 100 EUR a piece?
Please, could you try to do this with INTEL ATOM or CORE in this time for
this price? ;-)

Zoran Stojsavljevic

On Thu, Sep 7, 2017 at 2:23 AM, ron minnich  wrote:

> The original statement about BIOS -- that it was a second OS from the
> first day on -- is not correct.
>
> I am pretty sure the term BIOS (Basic Input Output Subsystem) comes from
> the early days, from CP/M. That's when I started hearing it anyway.
>
> The BIOS was the bottom half of CP/M. It provided an abstract device
> interface that the top half -- the part that came on a floppy -- could use.
> So the BIOS was not a second OS. It was 1/2 the OS you ran. Vendors created
> hardware and BIOS for the (usually 1024 byte) ROM and that ensured that
> CP/M could run on their box. The part of CP/M on the floppy, and the part
> in EEPROM, combined to provide a kernel. Kernel, of course, being a very
> loose term given the lack of processor modes.
>
> This was a very different model from, e.g., the minicomputers of the day,
> wherein the ROM was used to boot a kernel and then it got out of the way.
>
> So BIOS nature as a runtime entity was established very early on.
>
> Linux, for a long time, did not need no steenkin' BIOS (you can find that
> quote if you look back far enough). But those days are gone.
>
> ron
>
> On Wed, Sep 6, 2017 at 5:09 PM Julius Werner  wrote:
>
>> > The awkward thing about BIOS is that it was a second OS from the first
>> > day on – while the reasonable philosophy behind firmware should be:
>> > Start the board, load the OS and go back into your flash until reboot.
>>
>> My history lessons may be failing me here, but IIRC the main reason
>> for that was memory: DOS wasn't a full OS in the modern sense, it was
>> pretty much just a shell and a collection of utility programs. All the
>> actual device driving was done by the BIOS. So if you compare it with
>> a modern GNU/Linux machine, the BIOS was equivalent to Linux and DOS
>> to the GNU parts.
>>
>> The original IBM PC had so little memory that they couldn't really
>> afford to waste any of it on stuff like device drivers. So they put
>> the drivers in flash where they could be executed in-place, which
>> became the BIOS. Most of DOS itself (essentially the command.com
>> shell) was just unloaded whenever you launched a program and re-loaded
>> from disk afterwards, and while the program was running it mostly
>> interacted with the BIOS directly.
>>
>> So you're right that from our current point of view that having
>> callbacks into firmware makes little sense, but back then they were
>> working with what they had and it was pretty much the only way to get
>> as much out of the machine as they needed. The BIOS was really the
>> first OS on the platform, and all those later ones that implement
>> their own drivers (Windows, Linux) are breaking with the intended
>> paradigm.
>>
>> --
>> coreboot mailing list: coreboot@coreboot.org
>> https://mail.coreboot.org/mailman/listinfo/coreboot
>
>
> --
> coreboot mailing list: coreboot@coreboot.org
> https://mail.coreboot.org/mailman/listinfo/coreboot
>
-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[ron minnich]

The original statement about BIOS -- that it was a second OS from the first
day on -- is not correct.

I am pretty sure the term BIOS (Basic Input Output Subsystem) comes from
the early days, from CP/M. That's when I started hearing it anyway.

The BIOS was the bottom half of CP/M. It provided an abstract device
interface that the top half -- the part that came on a floppy -- could use.
So the BIOS was not a second OS. It was 1/2 the OS you ran. Vendors created
hardware and BIOS for the (usually 1024 byte) ROM and that ensured that
CP/M could run on their box. The part of CP/M on the floppy, and the part
in EEPROM, combined to provide a kernel. Kernel, of course, being a very
loose term given the lack of processor modes.

This was a very different model from, e.g., the minicomputers of the day,
wherein the ROM was used to boot a kernel and then it got out of the way.

So BIOS nature as a runtime entity was established very early on.

Linux, for a long time, did not need no steenkin' BIOS (you can find that
quote if you look back far enough). But those days are gone.

ron

On Wed, Sep 6, 2017 at 5:09 PM Julius Werner  wrote:

> > The awkward thing about BIOS is that it was a second OS from the first
> > day on – while the reasonable philosophy behind firmware should be:
> > Start the board, load the OS and go back into your flash until reboot.
>
> My history lessons may be failing me here, but IIRC the main reason
> for that was memory: DOS wasn't a full OS in the modern sense, it was
> pretty much just a shell and a collection of utility programs. All the
> actual device driving was done by the BIOS. So if you compare it with
> a modern GNU/Linux machine, the BIOS was equivalent to Linux and DOS
> to the GNU parts.
>
> The original IBM PC had so little memory that they couldn't really
> afford to waste any of it on stuff like device drivers. So they put
> the drivers in flash where they could be executed in-place, which
> became the BIOS. Most of DOS itself (essentially the command.com
> shell) was just unloaded whenever you launched a program and re-loaded
> from disk afterwards, and while the program was running it mostly
> interacted with the BIOS directly.
>
> So you're right that from our current point of view that having
> callbacks into firmware makes little sense, but back then they were
> working with what they had and it was pretty much the only way to get
> as much out of the machine as they needed. The BIOS was really the
> first OS on the platform, and all those later ones that implement
> their own drivers (Windows, Linux) are breaking with the intended
> paradigm.
>
> --
> coreboot mailing list: coreboot@coreboot.org
> https://mail.coreboot.org/mailman/listinfo/coreboot
-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Julius Werner]

> The awkward thing about BIOS is that it was a second OS from the first
> day on – while the reasonable philosophy behind firmware should be:
> Start the board, load the OS and go back into your flash until reboot.

My history lessons may be failing me here, but IIRC the main reason
for that was memory: DOS wasn't a full OS in the modern sense, it was
pretty much just a shell and a collection of utility programs. All the
actual device driving was done by the BIOS. So if you compare it with
a modern GNU/Linux machine, the BIOS was equivalent to Linux and DOS
to the GNU parts.

The original IBM PC had so little memory that they couldn't really
afford to waste any of it on stuff like device drivers. So they put
the drivers in flash where they could be executed in-place, which
became the BIOS. Most of DOS itself (essentially the command.com
shell) was just unloaded whenever you launched a program and re-loaded
from disk afterwards, and while the program was running it mostly
interacted with the BIOS directly.

So you're right that from our current point of view that having
callbacks into firmware makes little sense, but back then they were
working with what they had and it was pretty much the only way to get
as much out of the machine as they needed. The BIOS was really the
first OS on the platform, and all those later ones that implement
their own drivers (Windows, Linux) are breaking with the intended
paradigm.

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Zoran Stojsavljevic]

> Hint: on x86 it's essentially one billion instructions before you can
even think about using RAM, much less loading the payload.

This is why everybody using INTEL silicon do have FSP as A MUST (instead
SEC+PEI). And this is why CORE CPU creations (2C+) have > 500 million gates
per silicon. There gazillions of modes, which are probably only 10% used in
real scenarios. All others are testing modes. And INTEL made smart
investment forcing customers to pay for that (excessive silicon
use/overkill).

> which is to load linux from flash. It's working fine on UEFI today and
I'm seeing what it takes to replace the coreboot ramstage with linux.

YUP. Since no OS loader, only one/single OS instance to be actually loaded
from flash. Then you don't care about AHCI, SATA, IDE and other HDD/SSD
modes, also about GFX (embedded Linux) and also CSM or UEFI. No INT, no Run
Time Services. You are running true ram disk, unpacked from flash as well.

Zoran

On Tue, Sep 5, 2017 at 1:45 AM, ron minnich  wrote:

>
>
> On Mon, Sep 4, 2017 at 3:41 PM Philipp Stanner  wrote:
>
>>
>> Start the board, load the OS and go back into your flash until reboot.
>>
>
> Just checking, but have you looked at the code to see what "start the
> board" really means? Hint: on x86 it's essentially one billion instructions
> before you can even think about using RAM, much less loading the payload.
>
> I'm guessing, by what you are saying, that you are not familiar with what
> has to be done. Apologies if I am wrong :-)
>
> And, that said, I'm back to the old original idea, which is to load linux
> from flash. It's working fine on UEFI today and I'm seeing what it takes to
> replace the coreboot ramstage with linux.
>
> ron
>
> --
> coreboot mailing list: coreboot@coreboot.org
> https://mail.coreboot.org/mailman/listinfo/coreboot
>
-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[ron minnich]

On Mon, Sep 4, 2017 at 3:41 PM Philipp Stanner  wrote:

>
> Start the board, load the OS and go back into your flash until reboot.
>

Just checking, but have you looked at the code to see what "start the
board" really means? Hint: on x86 it's essentially one billion instructions
before you can even think about using RAM, much less loading the payload.

I'm guessing, by what you are saying, that you are not familiar with what
has to be done. Apologies if I am wrong :-)

And, that said, I'm back to the old original idea, which is to load linux
from flash. It's working fine on UEFI today and I'm seeing what it takes to
replace the coreboot ramstage with linux.

ron
-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Philipp Stanner]

Am Montag, den 04.09.2017, 20:15 + schrieb Peter Stuge:
>  legacy
> tables such as ACPI

ACPI is a open standard, isn't it?

> 
> The payload directly reads the filesystem from disk, no boot sector
> is used.

Indeed. So a payload built for cb won't try to call BIOS. And if
something after it tries it said payload should have implemented the
calls.

> coreboot with its payloads offers significant advantages over legacy
> BIOS and stillborn UEFI architectures.

C and open source is great. Though it frustrates me a bit that people
did all this work – and here we are again, still implementing BIOS-
Interrupts and all this old stuff. If it would compile on my machine I
would prefer a tiny loader like FILO.
The awkward thing about BIOS is that it was a second OS from the first
day on – while the reasonable philosophy behind firmware should be:
Start the board, load the OS and go back into your flash until reboot.

P.

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Peter Stuge]

Peter Stuge wrote:
> The MBR is irrelevant in the above scenario, it is never read.
..
> The payload directly reads the filesystem from disk, no boot sector is used.

To clarify, I mean code here. No code is read/loaded/used from MBR.

The partition table is read, but I expect that GRUB2 supports both
GPT and MBR partition tables, so MBR is not a requirement, and can
likely be changed at will, without needing to modify coreboot or
GRUB2 in boot flash, as long as the GRUB2 config file correctly
reflects system configuration.


//Peter

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Peter Stuge]

Zoran Stojsavljevic wrote:
> > But a bootloader built as a payload could also be built to use BIOS
> > interfaces. GRUB is one example of this.
> 
> Let us assume the following configuration:
> FSP -> Coreboot -> Payload: GRUB2 -> Linux
> 
> No legacy interrupts, correct?

Correct.

> So, what is this for the architecture? CSM? UEFI look alike?

Neither. There are no callbacks of any kind. Neither BIOS services
nor UEFI runtime services (which are like BIOS services, but much
more complicated for no reason other than inventor job security) are
present here.

coreboot does export a "coreboot tables" data structure in RAM, e.g. with
a memory map, and depending on build settings potentially also legacy
tables such as ACPI, which may required by the operating system.

coreboot tables contain only data, no code.


> I assume, the Linux image on the HDD/SSD was installed far before as CSM,
> so there is already existing MBR, and some CSM implemented legacy services
> (INTs) already exist, am I correct?

No.  The MBR is irrelevant in the above scenario, it is never read. 
GRUB2 uses its own disk and filesystem drivers to locate and load the
configured kernel and possibly initramfs files.

How the kernel and userland files got onto the disk drive is also
irrelevant. Beyond a recognized partition table, GRUB configuration
file and kernel files in the right directory on a supported
filesystem, nothing else matters.

The same is true for FILO.

The payload directly reads the filesystem from disk, no boot sector is used.


> I assume UEFI installed Linux would NOT run, since GRUB2 will be not able
> to find MBR... I assume GRUB2 as payload to Coreboot assumes MBR/Legacy
> (by default) implementation.

Again, it does not matter how Linux was installed, as long as GRUB2
can locate the kernel on the filesystem.


> Any comments on what I wrote here? ;-)

coreboot with its payloads offers significant advantages over legacy
BIOS and stillborn UEFI architectures.


//Peter

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Zoran Stojsavljevic]

> But a bootloader built as a payload could also be built to use BIOS
> interfaces. GRUB is one example of this.

Let us assume the following configuration:
FSP -> Coreboot -> Payload: GRUB2 -> Linux

No legacy interrupts, correct? So, what is this for the architecture? CSM?
UEFI look alike?

I assume, the Linux image on the HDD/SSD was installed far before as CSM,
so there is already existing MBR, and some CSM implemented legacy services
(INTs) already exist, am I correct?

I assume UEFI installed Linux would NOT run, since GRUB2 will be not able
to find MBR... I assume GRUB2 as payload to Coreboot assumes MBR/Legacy (by
default) implementation.

Any comments on what I wrote here? ;-)

Zoran

On Mon, Sep 4, 2017 at 12:01 PM, Philipp Stanner  wrote:

> Thanks so far. Very interesting.
>
> Am Montag, den 04.09.2017, 07:28 + schrieb Peter Stuge:
> >
> > coreboot itself can only start one payload, but SeaBIOS allows the
> > user to choose which of those payloads to start, in which case a
> > payload *does* have interrupt services available.
> >
> > Just to let you know that this is possible.
> >
>
> > There is no "32-bit instruction set". The instruction set is the
> > same,
> > there is no difference in what opcodes could be used (well except
> > maybe
> > gdt and idt stuff) but *addressing* is obviously different in 32-bit
> > mode,
> > and using 32-bit addressing makes everything a lot easier.
> >
>
> If there is no difference according the instruction set and as coreboot
> itself doesn't provide BIOS-services: Why don't we just switch the CPU
> to flat protected mode, do our duty and jump into the payload (SeaBIOS
> mainly) which then switches back to real mode if necessary, change the
> GDT or do whatever it needs to do?
> Switching back to Real Mode, using 32-Bit-addresses (what sounds like a
> hack to me. I can't imagine Intel intended this) is much more
> complicated than staying in PM, isn't it?
>
> x86 is pain. Sometimes I wonder why it even works at all. Let's hope
> there will be RISC-V-Boards one day.
>
> P.
>
> --
> coreboot mailing list: coreboot@coreboot.org
> https://mail.coreboot.org/mailman/listinfo/coreboot
>
-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Philipp Stanner]

Thanks so far. Very interesting.

Am Montag, den 04.09.2017, 07:28 + schrieb Peter Stuge:
> 
> coreboot itself can only start one payload, but SeaBIOS allows the
> user to choose which of those payloads to start, in which case a
> payload *does* have interrupt services available.
> 
> Just to let you know that this is possible.
> 

> There is no "32-bit instruction set". The instruction set is the
> same,
> there is no difference in what opcodes could be used (well except
> maybe
> gdt and idt stuff) but *addressing* is obviously different in 32-bit
> mode,
> and using 32-bit addressing makes everything a lot easier.
> 

If there is no difference according the instruction set and as coreboot
itself doesn't provide BIOS-services: Why don't we just switch the CPU
to flat protected mode, do our duty and jump into the payload (SeaBIOS
mainly) which then switches back to real mode if necessary, change the
GDT or do whatever it needs to do?
Switching back to Real Mode, using 32-Bit-addresses (what sounds like a
hack to me. I can't imagine Intel intended this) is much more
complicated than staying in PM, isn't it?

x86 is pain. Sometimes I wonder why it even works at all. Let's hope
there will be RISC-V-Boards one day.

P.

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Peter Stuge]

Philipp Stanner wrote:
> Once coreboot jumped into SeaBIOS-code the latter is responsible for
> providing the right interface for interrupt services.

Sure, coreboot does not provide interrupt services. I did not mean to
claim that it does. Sorry if there was confusion.


> I guess it doesn't matter very much in the end who switches the CPU
> to Real Mode.

One point I tried to make is that the CPU can very well be switched
back and forth several times before the actual OS kernel is running.


> My point is that coreboot's bios calls can't be called because they
> don't exist ^^

Sure, that's accurate, but having coreboot doesn't mean that there
are never BIOS interrupt services.


> > That depends. A *payload* can not assume interrupt services, but a
> > bootloader (e.g. in MBR) can very well assume interrupt services, 
> > especially since MBR is a BIOS paradigm.
> 
> Every payload built for and with coreboot won't try to call interrupts.

No payload is built with coreboot. Programs built as payloads will
indeed not call interrupt services, because they expect none.

But a bootloader built as a payload could also be built to use BIOS
interfaces. GRUB is one example of this.

And it is technically possible to put multiple payloads into CBFS.

coreboot itself can only start one payload, but SeaBIOS allows the
user to choose which of those payloads to start, in which case a
payload *does* have interrupt services available.

Just to let you know that this is possible.


> > > And you can't use the 32-Bit-Instructionset anymore, can you?
> > 
> > That's orthogonal. You can use 32-bit instructions in 16-bit mode and
> > vice versa. The 0x66 and 0x67 opcode prefixes can be used to set an
> > instruction mode for a single instruction.
> 
> But in the beginning of this thread on 02 Aug Ron Minnich wrote
> that coreboot switches to PM, because: 
> 
> "And we want the 32-bit instruction set."

Maybe he didn't consider the prefix, or maybe he was thinking about
how GCC+binutils are pretty poor at dealing with the different modes
and swapping between them.

On the instruction level it's not a big deal to explicitly set the
"other" mode, but the toolchain is not very strong when it comes to
16-bit, making it more difficult to do so when writing C code. It was
created for the operating system environment and it shows, but that's
perfectly fine.

There is no "32-bit instruction set". The instruction set is the same,
there is no difference in what opcodes could be used (well except maybe
gdt and idt stuff) but *addressing* is obviously different in 32-bit mode,
and using 32-bit addressing makes everything a lot easier.


//Peter

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Philipp Stanner]

Am Samstag, den 02.09.2017, 12:36 + schrieb Peter Stuge:
> 
> Sure, but a payload can. SeaBIOS aims to provide a complete BIOS with
> all neccessary interrupt services for legacy compatibility.

Once coreboot jumped into SeaBIOS-code the latter is responsible for
providing the right interface for interrupt services.

I guess it doesn't matter very much in the end who switches the CPU to
Real Mode. My point is that coreboot's bios calls can't be called
because they don't exist ^^



> 
> That depends. A *payload* can not assume interrupt services, but a
> bootloader (e.g. in MBR) can very well assume interrupt services, 
> especially since MBR is a BIOS paradigm.

Every payload built for and with coreboot won't try to call interrupts.

> 
> > And you can't use the 32-Bit-Instructionset anymore, can you?
> 
> That's orthogonal. You can use 32-bit instructions in 16-bit mode and
> vice versa. The 0x66 and 0x67 opcode prefixes can be used to set an
> instruction mode for a single instruction.

But in the beginning of this thread on 02 Aug Ron Minnich wrote that
coreboot switches to PM, because: 

"And we want the 32-bit instruction set."

P.

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Peter Stuge]

Philipp Stanner wrote:
> > > Why would I want to address memory in RM with 32 Bits? I don't see
> > > any difference to using PM without Paging enabled.
> > 
> > In a bootloader (after coreboot) you often want to call BIOS
> > interrupt services which assume real mode, because that was the
> > only mode, when the interrupt services were created.
> > 
> > For coreboot itself, I think there is little if any advantage to big
> 
> Wait. Coreboot doesn't implement BIOS-Interrupt-Services.

Sure, but a payload can. SeaBIOS aims to provide a complete BIOS with
all neccessary interrupt services for legacy compatibility.


> So no payload started by cb should ever try to call BIOS, should it?

That depends. A *payload* can not assume interrupt services, but a
bootloader (e.g. in MBR) can very well assume interrupt services, 
especially since MBR is a BIOS paradigm.


> And you can't use the 32-Bit-Instructionset anymore, can you?

That's orthogonal. You can use 32-bit instructions in 16-bit mode and
vice versa. The 0x66 and 0x67 opcode prefixes can be used to set an
instruction mode for a single instruction.


//Peter

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Philipp Stanner]

Am Donnerstag, den 03.08.2017, 12:48 + schrieb Peter Stuge:
> Philipp Stanner wrote:
> > Why would I want to address memory in RM with 32 Bits? I don't see
> > any difference to using PM without Paging enabled.
> 
> In a bootloader (after coreboot) you often want to call BIOS
> interrupt services which assume real mode, because that was the
> only mode, when the interrupt services were created.
> 
> For coreboot itself, I think there is little if any advantage to big
> real mode.
> 
> 
> //Peter

Wait. Coreboot doesn't implement BIOS-Interrupt-Services.
So no payload started by cb should ever try to call BIOS, should it?

And you can't use the 32-Bit-Instructionset anymore, can you?

P.

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Julius Werner]

>
> Unfortunately, this picture represents ONLY 32bit transitions, there are
> two more modes for 64 bit, not shown here. If I find the representative,
> I'll post it on the forum (@ list).
>

The box labeled "IA-32e Mode" is 64-bit mode (in their infinite wisdom,
Intel chose a name containing the number 32 for their copy of amd64 mode).
I think the other mode you're referring to (32-bit compatibility mode for
64-bit systems) is technically considered a "sub-mode" of that, so that's
probably why they only have one box to represent both. Compatibility mode
is controlled by a bit in the CS descriptor if I recall correctly.
-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[ron minnich]

in the akaros virtual machine code, we set up a simple 1:1 map and start
linux at the 64-bit entry point, at which point it builds its own page
tables. So entering a payload in long mode is certainly possible and IMHO
ought to be the standard on amd64 -- not that anyone cares what I think
anyway :-)

Anyway, I'm going to be working on replacing ramstage with linux again, so
I'm not as concerted with coreboot ramstage just now ;-)

On Thu, Aug 3, 2017 at 9:16 AM Zoran Stojsavljevic <
zoran.stojsavlje...@gmail.com> wrote:

> OK Ron,
>
> I needed some time to reconcile my thoughts. So,as Aaron pointed, it is
> impossible to do any long mode paging in ROM stage.
>
> And here we come to the Mystery of CB Organism (h... It is very close
> to some other term). ;-)
>
> Now, here is how it will play for you and Coreboot, after ROM stage (I'll
> call it RON stage, since it is RAM stage, divided in two sections).
>
> Why RON? Ready Or Not, that is the question!
>
> So RON stage = RAM1 stage + RAM2 stage.
>
> RAM1 stage: Since some RAM Physical mapping (AFTER CRB) MUST be done at
> the beginning of the RAM stage, lookalike BIOS one (in PEI phase, I
> assume), CB will proceed with the usual activities, set in Protected Mode.
> And, as we know so far how it works, it'll finish this phase, looking for
> Payload.
>
> Now, instead of payload stage, here we go with the RAM2 stage. Now the CB
> will switch to the Long Mode (flat x86_64), addressing at maximum of 48
> bits of the physically available CPU address memory bus - 256 TB of DDRAM
> memory (I'll drop here 64 bit PAE extension (additional 4 bits of physical
> address space), for clarity).
>
> Here, you'll start initializing MMU, with 4 TLB tables: CR3 -> PML4 (512
> entries) -> PDTP (512 entries) -> PDT (512 entries) -> PT (512 entries +
> last 12 bits freed for the MMU attributes). So, if your HW have in average
> 8GB of memory, you'll need ONLY 1 entry in PML4, 8 entries in PDTP -> and
> full tables for these 8 entries. I see that you'll do NOT a Virtualization,
> rather a logical address shift for the constant address offset. With proper
> init of the attributes in the PE tables, per 4KB page - your finest
> granularity).
>
> And, YES, you'll have your memory protection in full swing (just for the
> de-referencing the NULL pointers), since you do NOT have more that one
> process working, in linear mode (so the bus error will appear ONLY for NULL
> pointers).
>
> Now, after you've done this, what is the next step, somebody will ask?
> [A] Reverting to Protected Mode prior Payload stage?
> [B] Proceeding with the Long Mode passing thread of execution to
> payloads... Which need to operate in Long Mode!? Really???
> [C] Reverting to Virtual x86 Mode???
> [D] Reverting to Real Mode???
>
> Seems that you will do scenario [A]. But if you choose to have payload as
> 64b Linux [B], then I see where you are pointing/going with this
> approach... To fast Server (?!) boot-ups, but you need to reconsider, since
> 64b Linux still will reconfigure MMU, doesn't it???
>
> Or you would like to skip kernel MMU init while booting 64b kernel.
> Letting kernel to pick up (already) hot MMU, on fly?
>
> As far, it'll be: Boot stage + ROM stage + RON stage (RAM1 stage in
> Protected Mode, + RAM2 stage in 64b Long Mode) + 64b kernel payload. Solely
> out of FLASH (smells as DOD MIL RT applications, drones, unmanned planes,
> missiles)... Interesting scenario, anyway!
>
> (Mysteries of CB organisms) ;-)
>
> Zoran
>
> On Wed, Aug 2, 2017 at 9:45 PM, ron minnich  wrote:
>
>> right, Aaron, you keep reminding me of that and I keep forgetting it :-)
>>
>> so, ok, ramstage it is for paging. The x86 is so darned annoying. We had
>> paging in the bootblock for PPC on linuxbios.
>>
>> On Wed, Aug 2, 2017 at 12:43 PM Aaron Durbin  wrote:
>>
>>> On Wed, Aug 2, 2017 at 1:36 PM, ron minnich  wrote:
>>> >
>>> >
>>> > On Wed, Aug 2, 2017 at 12:08 PM Zoran Stojsavljevic
>>> >  wrote:
>>> >>
>>> >>
>>> >>
>>> >> So, turning on the Virtual Mode (paging ON), you'll also imply that
>>> >> Coreboot will initialize and use MMU, don't you?
>>> >
>>> >
>>> > sure.
>>> >
>>> >>
>>> >>
>>> >> Am I correct, or you can use paging without MMU?!
>>> >
>>> >
>>> > MMU must be on.
>>> >
>>> >>
>>> >> I doubt that, perhaps. So, then, I would like to see this code in
>>> Coreboot
>>> >> where MMU is initialized. And how it is initialized (maybe there is a
>>> some
>>> >> over-simplified mode of operation using 2MB pages).
>>> >
>>> >
>>> > yep.
>>> >
>>> >>
>>> >>
>>> >> And my question is: what for? Or I did not get the idea... Who really
>>> >> needs to use paging in boot-loaders? Even INTEL, which (on purpose)
>>> makes
>>> >> things way over-dimensioned and over-complicated, does NOT use paging
>>> in
>>> >> UEFI BIOSes, so far???
>>> >>
>>> >
>>> > it's trivial to turn on paging. In harvey I've got it down to 160
>>> lines,
>>> > including comments and white space. I made it much more compact in my
>>> CL
>>> > fr

Re: [coreboot] About Paging, Realmode and what is going on

[Zoran Stojsavljevic]

OK Ron,

I needed some time to reconcile my thoughts. So,as Aaron pointed, it is
impossible to do any long mode paging in ROM stage.

And here we come to the Mystery of CB Organism (h... It is very close
to some other term). ;-)

Now, here is how it will play for you and Coreboot, after ROM stage (I'll
call it RON stage, since it is RAM stage, divided in two sections).

Why RON? Ready Or Not, that is the question!

So RON stage = RAM1 stage + RAM2 stage.

RAM1 stage: Since some RAM Physical mapping (AFTER CRB) MUST be done at the
beginning of the RAM stage, lookalike BIOS one (in PEI phase, I assume), CB
will proceed with the usual activities, set in Protected Mode. And, as we
know so far how it works, it'll finish this phase, looking for Payload.

Now, instead of payload stage, here we go with the RAM2 stage. Now the CB
will switch to the Long Mode (flat x86_64), addressing at maximum of 48
bits of the physically available CPU address memory bus - 256 TB of DDRAM
memory (I'll drop here 64 bit PAE extension (additional 4 bits of physical
address space), for clarity).

Here, you'll start initializing MMU, with 4 TLB tables: CR3 -> PML4 (512
entries) -> PDTP (512 entries) -> PDT (512 entries) -> PT (512 entries +
last 12 bits freed for the MMU attributes). So, if your HW have in average
8GB of memory, you'll need ONLY 1 entry in PML4, 8 entries in PDTP -> and
full tables for these 8 entries. I see that you'll do NOT a Virtualization,
rather a logical address shift for the constant address offset. With proper
init of the attributes in the PE tables, per 4KB page - your finest
granularity).

And, YES, you'll have your memory protection in full swing (just for the
de-referencing the NULL pointers), since you do NOT have more that one
process working, in linear mode (so the bus error will appear ONLY for NULL
pointers).

Now, after you've done this, what is the next step, somebody will ask?
[A] Reverting to Protected Mode prior Payload stage?
[B] Proceeding with the Long Mode passing thread of execution to
payloads... Which need to operate in Long Mode!? Really???
[C] Reverting to Virtual x86 Mode???
[D] Reverting to Real Mode???

Seems that you will do scenario [A]. But if you choose to have payload as
64b Linux [B], then I see where you are pointing/going with this
approach... To fast Server (?!) boot-ups, but you need to reconsider, since
64b Linux still will reconfigure MMU, doesn't it???

Or you would like to skip kernel MMU init while booting 64b kernel. Letting
kernel to pick up (already) hot MMU, on fly?

As far, it'll be: Boot stage + ROM stage + RON stage (RAM1 stage in
Protected Mode, + RAM2 stage in 64b Long Mode) + 64b kernel payload. Solely
out of FLASH (smells as DOD MIL RT applications, drones, unmanned planes,
missiles)... Interesting scenario, anyway!

(Mysteries of CB organisms) ;-)

Zoran

On Wed, Aug 2, 2017 at 9:45 PM, ron minnich  wrote:

> right, Aaron, you keep reminding me of that and I keep forgetting it :-)
>
> so, ok, ramstage it is for paging. The x86 is so darned annoying. We had
> paging in the bootblock for PPC on linuxbios.
>
> On Wed, Aug 2, 2017 at 12:43 PM Aaron Durbin  wrote:
>
>> On Wed, Aug 2, 2017 at 1:36 PM, ron minnich  wrote:
>> >
>> >
>> > On Wed, Aug 2, 2017 at 12:08 PM Zoran Stojsavljevic
>> >  wrote:
>> >>
>> >>
>> >>
>> >> So, turning on the Virtual Mode (paging ON), you'll also imply that
>> >> Coreboot will initialize and use MMU, don't you?
>> >
>> >
>> > sure.
>> >
>> >>
>> >>
>> >> Am I correct, or you can use paging without MMU?!
>> >
>> >
>> > MMU must be on.
>> >
>> >>
>> >> I doubt that, perhaps. So, then, I would like to see this code in
>> Coreboot
>> >> where MMU is initialized. And how it is initialized (maybe there is a
>> some
>> >> over-simplified mode of operation using 2MB pages).
>> >
>> >
>> > yep.
>> >
>> >>
>> >>
>> >> And my question is: what for? Or I did not get the idea... Who really
>> >> needs to use paging in boot-loaders? Even INTEL, which (on purpose)
>> makes
>> >> things way over-dimensioned and over-complicated, does NOT use paging
>> in
>> >> UEFI BIOSes, so far???
>> >>
>> >
>> > it's trivial to turn on paging. In harvey I've got it down to 160 lines,
>> > including comments and white space. I made it much more compact in my CL
>> > from 1/2014.
>> >
>> > If you turn on paging you can capture null pointer derefs, and all
>> other out
>> > of range address references. And relocatable modules are much easier..
>> >
>> > My question is, why not do it? I don't see any harm. All the other
>> platforms
>> > in coreboot use paging. What's so special about x86 that it should not
>> use
>> > paging?
>>
>> You can't use it for CAR so that sorta throws out romstage and before.
>> It's up to the implementation of how CAR works within the hierarchy,
>> but with hardware page walkers that pull things through the cache
>> things can go badly. So then you could enable paging for ramstage.
>> Need to locate storage for the page tables, but t

Re: [coreboot] About Paging, Realmode and what is going on

[Peter Stuge]

Philipp Stanner wrote:
> Why would I want to address memory in RM with 32 Bits? I don't see
> any difference to using PM without Paging enabled.

In a bootloader (after coreboot) you often want to call BIOS
interrupt services which assume real mode, because that was the
only mode, when the interrupt services were created.

For coreboot itself, I think there is little if any advantage to big real mode.


//Peter

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[ron minnich]

On Wed, Aug 2, 2017 at 1:38 PM Philipp Stanner  wrote:

> Dear Patrick, dear Zoran & List,
>
> thank you, this was *very* helpful. I had some misunderstandings regarding
> function and features of the CPU-modes.
>
> Let me sum it up again and feel free to correct further mistakes.
>
>- 16-Bit-Real Mode: No virtual memory, no segmentation. 2^20 addresses
>of memory.
>
>
and really dumb segmentation to get that 20 bits: 16 bit segment, 16 bit
offset, 20 bit address from (seg << 4) + offset

So a given address in real mode can be created many, many different ways:
f000:0 and :0 are the same address. The horror. I saw a talk once
by the inventor of this mess and he was actually proud of it. Go figure.


>- Protected Mode (flat mode?): CPU is in Protected Mode, Paging is
>off. The primary difference to Real Mode is that 2^32 Bytes of memory are
>available. Coreboot + Payloads use this mode, because they need >1MiB of
>memory due to various reasons
>
>
And we want the 32-bit instruction set. Also, flat is not really the right
word. You can have a segment descriptor that has a built in offset and a
built in limit such that, e.g., you can only address 20 bits of the 32 bit
address space starting at, e.g., 0x2000. I think you would find a reading
of how the descriptors work very interesting.

Addresses in protected mode are virtual addresses, and are mapped to
physical addresses as determined by the segment used.


>
>- Virtual Mode: Paging is on, the MMU is on. Memory can only be
>accessed by using virtual addresses. Details are specified in the GDT.
>
>
no. The GDT is about segments, i.e. protected mode. I don't like the word
virtual here unless you want to say virtual address. When I've used the
word virtual before on the list, I've gotten in trouble, because people
associate virtual memory with swapping pages to a disk. Just call it paged
mode and there's less confusion.


>
>- Long Mode: Paged virtual 64-Bit mode. Standard mode of modern
>operating systems (while I presume Protected Virtual Mode is the Standard
>for any OS on i386)
>
>
pretty much.

thanks

ron
-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Philipp Stanner]

Dear Patrick, dear Zoran & List,

thank you, this was *very* helpful. I had some misunderstandings
regarding function and features of the CPU-modes.

Let me sum it up again and feel free to correct further mistakes.

  * 16-Bit-Real Mode: No virtual memory, no segmentation. 2^20 addresses
of memory.
  * Protected Mode (flat mode?): CPU is in Protected Mode, Paging is
off. The primary difference to Real Mode is that 2^32 Bytes of
memory are available. Coreboot + Payloads use this mode, because
they need >1MiB of memory due to various reasons
  * Virtual Mode: Paging is on, the MMU is on. Memory can only be
accessed by using virtual addresses. Details are specified in the GDT.
  * Long Mode: Paged virtual 64-Bit mode. Standard mode of modern
operating systems (while I presume Protected Virtual Mode is the
Standard for any OS on i386)


Am 02.08.2017 um 18:20 schrieb Zoran Stojsavljevic:
>
>
> [4] Once the OS boot loader takes over (Like GRUB), it'll start OS,
> which will switch from the Protected to the Virtual Mode (using MMU),
> Then paging will take places, and each process will have its own set
> of tables, and its own initial value for CR3 (when process context
> switches). I warmly suggest to you to inspect (bit by bit) CR1
> register, since this one is crucial/essential for introducing these modes.
Thanks, I will do that. I intend to play around with QEMU and maybe have
a closer look to Intel's Programmer Manuals.

But I hope at least one assumption is right: Once coreboot jumped into
the payload no coreboot-code will be executed anymore. And once the
payload jumped into Linux no payload-code will be executed anymore?
Meaning the payload takes complete control over RAM and CPU.

Traditional BIOS (so I very much expect SeaBIOS to do the same) stays
somewhere within the 16-bit-address-space, even when Windows or Linux
are running, with it's Interrupt Service Rutines ready to do stuff.

Greetings
P.
-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[ron minnich]

right, Aaron, you keep reminding me of that and I keep forgetting it :-)

so, ok, ramstage it is for paging. The x86 is so darned annoying. We had
paging in the bootblock for PPC on linuxbios.

On Wed, Aug 2, 2017 at 12:43 PM Aaron Durbin  wrote:

> On Wed, Aug 2, 2017 at 1:36 PM, ron minnich  wrote:
> >
> >
> > On Wed, Aug 2, 2017 at 12:08 PM Zoran Stojsavljevic
> >  wrote:
> >>
> >>
> >>
> >> So, turning on the Virtual Mode (paging ON), you'll also imply that
> >> Coreboot will initialize and use MMU, don't you?
> >
> >
> > sure.
> >
> >>
> >>
> >> Am I correct, or you can use paging without MMU?!
> >
> >
> > MMU must be on.
> >
> >>
> >> I doubt that, perhaps. So, then, I would like to see this code in
> Coreboot
> >> where MMU is initialized. And how it is initialized (maybe there is a
> some
> >> over-simplified mode of operation using 2MB pages).
> >
> >
> > yep.
> >
> >>
> >>
> >> And my question is: what for? Or I did not get the idea... Who really
> >> needs to use paging in boot-loaders? Even INTEL, which (on purpose)
> makes
> >> things way over-dimensioned and over-complicated, does NOT use paging in
> >> UEFI BIOSes, so far???
> >>
> >
> > it's trivial to turn on paging. In harvey I've got it down to 160 lines,
> > including comments and white space. I made it much more compact in my CL
> > from 1/2014.
> >
> > If you turn on paging you can capture null pointer derefs, and all other
> out
> > of range address references. And relocatable modules are much easier..
> >
> > My question is, why not do it? I don't see any harm. All the other
> platforms
> > in coreboot use paging. What's so special about x86 that it should not
> use
> > paging?
>
> You can't use it for CAR so that sorta throws out romstage and before.
> It's up to the implementation of how CAR works within the hierarchy,
> but with hardware page walkers that pull things through the cache
> things can go badly. So then you could enable paging for ramstage.
> Need to locate storage for the page tables, but that's not an
> impossible thing either.
>
> >
> > ron
> >
> > --
> > coreboot mailing list: coreboot@coreboot.org
> > https://mail.coreboot.org/mailman/listinfo/coreboot
>
-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Aaron Durbin via coreboot]

On Wed, Aug 2, 2017 at 1:36 PM, ron minnich  wrote:
>
>
> On Wed, Aug 2, 2017 at 12:08 PM Zoran Stojsavljevic
>  wrote:
>>
>>
>>
>> So, turning on the Virtual Mode (paging ON), you'll also imply that
>> Coreboot will initialize and use MMU, don't you?
>
>
> sure.
>
>>
>>
>> Am I correct, or you can use paging without MMU?!
>
>
> MMU must be on.
>
>>
>> I doubt that, perhaps. So, then, I would like to see this code in Coreboot
>> where MMU is initialized. And how it is initialized (maybe there is a some
>> over-simplified mode of operation using 2MB pages).
>
>
> yep.
>
>>
>>
>> And my question is: what for? Or I did not get the idea... Who really
>> needs to use paging in boot-loaders? Even INTEL, which (on purpose) makes
>> things way over-dimensioned and over-complicated, does NOT use paging in
>> UEFI BIOSes, so far???
>>
>
> it's trivial to turn on paging. In harvey I've got it down to 160 lines,
> including comments and white space. I made it much more compact in my CL
> from 1/2014.
>
> If you turn on paging you can capture null pointer derefs, and all other out
> of range address references. And relocatable modules are much easier..
>
> My question is, why not do it? I don't see any harm. All the other platforms
> in coreboot use paging. What's so special about x86 that it should not use
> paging?

You can't use it for CAR so that sorta throws out romstage and before.
It's up to the implementation of how CAR works within the hierarchy,
but with hardware page walkers that pull things through the cache
things can go badly. So then you could enable paging for ramstage.
Need to locate storage for the page tables, but that's not an
impossible thing either.

>
> ron
>
> --
> coreboot mailing list: coreboot@coreboot.org
> https://mail.coreboot.org/mailman/listinfo/coreboot

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[ron minnich]

On Wed, Aug 2, 2017 at 12:08 PM Zoran Stojsavljevic <
zoran.stojsavlje...@gmail.com> wrote:

>
>
> So, turning on the Virtual Mode (paging ON), you'll also imply that
> Coreboot will initialize and use MMU, don't you?
>

sure.


>
> Am I correct, or you can use paging without MMU?!
>

MMU must be on.


> I doubt that, perhaps. So, then, I would like to see this code in Coreboot
> where MMU is initialized. And how it is initialized (maybe there is a some
> over-simplified mode of operation using 2MB pages).
>

yep.


>
> And my question is: what for? Or I did not get the idea... Who really
> needs to use paging in boot-loaders? Even INTEL, which (on purpose) makes
> things way over-dimensioned and over-complicated, does NOT use paging in
> UEFI BIOSes, so far???
>
>
it's trivial to turn on paging. In harvey I've got it down to 160 lines,
including comments and white space. I made it much more compact in my CL
from 1/2014.

If you turn on paging you can capture null pointer derefs, and all other
out of range address references. And relocatable modules are much easier..

My question is, why not do it? I don't see any harm. All the other
platforms in coreboot use paging. What's so special about x86 that it
should not use paging?

ron
-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Patrick Georgi via coreboot]

2017-08-02 21:08 GMT+02:00 Zoran Stojsavljevic :
> And my question is: what for? Or I did not get the idea... Who really needs
> to use paging in boot-loaders? Even INTEL, which (on purpose) makes things
> way over-dimensioned and over-complicated, does NOT use paging in UEFI
> BIOSes, so far???
They must for X64 builds. Long Mode _requires_ page tables (even if
it's a single 1TB page to create a 1:1 mapping).

But fear not, UEFI also supported paging for a _really_ long time for
Runtime Services, so everything DXE onward better prepares for being
called by Runtime Services or SMM, even if the spec explicitly says
that it shouldn't happen - it did. Apparently they just couldn't fully
eschew the mantle of superfluous complexity.


Patrick
-- 
Google Germany GmbH, ABC-Str. 19, 20354 Hamburg
Registergericht und -nummer: Hamburg, HRB 86891, Sitz der Gesellschaft: Hamburg
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Zoran Stojsavljevic]

Hey Ron,

> You can't enter long mode (x86-64) mode without enabling paging. you need
long mode if you need to operate on more than 2^36 bytes
> of memory. That's coming everywhere. Hence,* the future of coreboot is
unambiguously one in which paging is on and at least*
*> in my case** I found it's possible to make that happen in the boot block
*-- albeit a bigger boot block than we have today.

So, turning on the Virtual Mode (paging ON), you'll also imply that
Coreboot will initialize and use MMU, don't you?

Am I correct, or you can use paging without MMU?! I doubt that, perhaps.
So, then, I would like to see this code in Coreboot where MMU is
initialized. And how it is initialized (maybe there is a some
over-simplified mode of operation using 2MB pages). Since... You know,
32bit OSes use physical pages to map them via MMU to be virtual contiguous
space (0-4GB space per process). And since CB is still 32bit creation... In
other words, you would like to have ONLY one uninterruptible "process" in
CB running all the time (no threading at all) to be virtualized???

And my question is: what for? Or I did not get the idea... Who really needs
to use paging in boot-loaders? Even INTEL, which (on purpose) makes things
way over-dimensioned and over-complicated, does NOT use paging in UEFI
BIOSes, so far???

Zoran

On Wed, Aug 2, 2017 at 7:15 PM, ron minnich  wrote:

> I'll try not to mess this explanation up too much, but it's partly from
> the docs and partly what we've learned over the years. I am assuming that
> you've at least looked at the coreboot boot block and read some of the x86
> docs, although from this note and previous notes you have written I'm still
> not sure. Have you done some of your homework :-)
>
> The architecture definition of x86 is that power on reset starts up at
> 0x0 in what looks to me like big real mode (a.k.a. "unreal mode" among
> many other names). From using JTAG debuggers over the years it's been clear
> to me anyway that it's not exactly classic x86 16-bit mode. Coreboot, like
> most firmware, does a classic 16-bit longjmp with segment:offset to get to
> the coreboot bootblock. This longjmp does flip us into classic x86 mode (as
> is very apparent from JTAG debuggers). Coreboot in about 12 instructions
> flips to 32-bit protected mode (no paging however). Then we go to C.
>
> FWIW, many vendors told us at the time it was impossible to write firmware
> in C, as we did in 2000, so it was nice to show it could be done.
>
> As for paging: we've had several platforms, starting in 2000, that used
> paging (which does not always imply page tables). Alpha comes to mind: we
> had a basic PALCode that implemented 1:1 mapping. Most coreboot platforms
> use paging.
>
> I note that some amount of paging is now in coreboot for x86 and maybe
> someone can let us know where and when it's used.
>
> But as for real mode, no, starting from POR in protected mode has never
> been an option intel was willing to support. We've asked ("we" not being
> Google, but from my earlier life in HPC). There's no technical reason it
> could not work (it's microcode, after all) but it would be a pretty big
> change from the "party like it's 1978" model of the x86.
>
> Note that many CPUs, such PPC starting ca. 2008, can not run with paging
> off; they start up with default entries in the TLB and paging is on from
> power on reset.
>
> You can't enter long mode (x86-64) mode without enabling paging. you need
> long mode if you need to operate on more than 2^36 bytes of memory. That's
> coming everywhere. Hence, the future of coreboot is unambiguously one in
> which paging is on and at least in my case I found it's possible to make
> that happen in the boot block -- albeit a bigger boot block than we have
> today.
>
> ron
>
-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[ron minnich]

I'll try not to mess this explanation up too much, but it's partly from the
docs and partly what we've learned over the years. I am assuming that
you've at least looked at the coreboot boot block and read some of the x86
docs, although from this note and previous notes you have written I'm still
not sure. Have you done some of your homework :-)

The architecture definition of x86 is that power on reset starts up at
0x0 in what looks to me like big real mode (a.k.a. "unreal mode" among
many other names). From using JTAG debuggers over the years it's been clear
to me anyway that it's not exactly classic x86 16-bit mode. Coreboot, like
most firmware, does a classic 16-bit longjmp with segment:offset to get to
the coreboot bootblock. This longjmp does flip us into classic x86 mode (as
is very apparent from JTAG debuggers). Coreboot in about 12 instructions
flips to 32-bit protected mode (no paging however). Then we go to C.

FWIW, many vendors told us at the time it was impossible to write firmware
in C, as we did in 2000, so it was nice to show it could be done.

As for paging: we've had several platforms, starting in 2000, that used
paging (which does not always imply page tables). Alpha comes to mind: we
had a basic PALCode that implemented 1:1 mapping. Most coreboot platforms
use paging.

I note that some amount of paging is now in coreboot for x86 and maybe
someone can let us know where and when it's used.

But as for real mode, no, starting from POR in protected mode has never
been an option intel was willing to support. We've asked ("we" not being
Google, but from my earlier life in HPC). There's no technical reason it
could not work (it's microcode, after all) but it would be a pretty big
change from the "party like it's 1978" model of the x86.

Note that many CPUs, such PPC starting ca. 2008, can not run with paging
off; they start up with default entries in the TLB and paging is on from
power on reset.

You can't enter long mode (x86-64) mode without enabling paging. you need
long mode if you need to operate on more than 2^36 bytes of memory. That's
coming everywhere. Hence, the future of coreboot is unambiguously one in
which paging is on and at least in my case I found it's possible to make
that happen in the boot block -- albeit a bigger boot block than we have
today.

ron
-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

[coreboot] About Paging, Realmode and what is going on

[ingegneriafore...@alice.it]

Hello everybody,

following the mailing list about the questions posed by Philipp Stanner I read:

"Why does every modern CPU still start in RM?

Many industries run on DOS. Many system developers have created
in-house BIOS extensions. x86 will never fully lose its 16-bit legacy."

I'm agree with this answer, and I also understand that some legacy MUST be 
maintained for the backward HW compatibility, as Zoran explains.

However i wonder if  is there the possibility that Coreboot boots directly and 
exclusively in protected mode. Is it configurable this behaviour in coreboot ? 
That is: is it possible avoid the x86 boots in Real Mode, "forgotting" to be 
compatible with old hardware ?

Best Regards.

Vincenzo.


Forensic Consultant
Tribunale di Lecce

Studio: Strada di Garibaldi - Contrada Paradisi
73010 Lequile (LE)

cell: 339.7968555
skype: vincenzo.di_salvo
-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Zoran Stojsavljevic]

Hello Philipp,

There are lot of confusion you have created with your naming convention...
So we need here to "Divide and Concur", since this thread really remains me
of Spaghetti Bologneze.

Here is the partial picture how x86 operates:

[image: Inline image 1]

So you see here three modes, you are talking about. Every INTEL CPU starts
in Real Mode, and then switches to the protected mode.

So, you have some legacy you MUST maintain for the backward HW
compatibility.

As Patrick Georgi very correctly mentioned, let me recap what he said, with
my additions, to be much clearer with the picture I have provided.

[1]  Every x86 CPU starts in Real Mode, for the legacy reasons. the Lecacy
mode must be maintained (Real Mode replicates with the VERY first PC XT far
back to 1980, with some first 1MB memory mapping improvements). The
BackWard HW compatibility was maintained for years, and it went too far
that it can be changed over the nite. I am not going to explain why: too
long it takes.

[2] CB operates in Protected Mode, WITHOUT Paging/Virtual Mode switched on,
since this mode is ONLY used with MMU (which is HUGE HW extention to create
Virtual use of the Memory), which is managed by true OS (like Linux, WIN,
QNX, MAC OS, VMS, you name it). CoreBoot, U-Boot, BIOS boot-loaders: they
all operate from the flat/Protected Mode, and Patrick explained this very
well/excellent. In addition, I must say that this first 4GB MUST be
assigned to the benefit of the underlying HW (PCIe, GFX, SMM mode, Flash
region, ACPI tables, and so on). This assignment is done strictly
Physically, with the CPU HW helping for some regions for remapping them
(since CPU does see what CB does for it as Physical memory assignment, but
MMU [when OS runs] sees it quite differently)!

[3] As for payloads (also working in Protected Mode), there are some which
will make their own definitions of the memory, some/most which will work
deeper (as SeaBIOS, for example) on the CB Physical memory definition. As
well as these payloads (depending upon what the usage is) will add more
drivers for the certain devices, CB did not add in the first place. For
example, SeaBIOS adds IDE (maybe even AHCI, not sure) driver to this set.
Some people can define their own payloads with the set of specific drivers,
depending upon what the HW platform is intended for.

[4] Once the OS boot loader takes over (Like GRUB), it'll start OS, which
will switch from the Protected to the Virtual Mode (using MMU), Then paging
will take places, and each process will have its own set of tables, and its
own initial value for CR3 (when process context switches). I warmly suggest
to you to inspect (bit by bit) CR1 register, since this one is
crucial/essential for introducing these modes.

I really hope this helps (I've tried to develop some systematic approach to
the topic). The booting process will go to stages, and once CPU abandons
Real Mode, it'll (generally) not return to it (only in some special cases
such as SMM Mode). Once It abandons Protected Mode, it'll stay in Virtual
Mode, and so on. But backward transitions are possible, as you see from the
picture.

Zoran

On Mon, Jul 31, 2017 at 10:52 AM, Philipp Stanner  wrote:

> Dear folks and techpriests,
>
> the more I want to contribute and learn about low-level-code the less I
> understand, it seems.
>
>
>1. cb switches the CPU immediately to Protected Mode, yet Payloads
>like seaBIOS work in Real Mode. Does coreboot switch the CPU always back to
>RM before jumping to the payload?
>2. When CB switches to PM - who generates and administrates the Page
>Tables and where?
>3. Gustavo Duarte writes
> that
>GRUB switches from protected mode to real mode and vice versa all the time
>to address >1MiB of RAM and also use the BIOS-calls. If this is true using
>GRUB as payload would not work, as GRUB needs to call the non-existent
>BIOS, right?
>4. Once CB is in PM it can't access physical addresses anymore? It
>doesn't need to, too?
>5. PM means RAM-access is only possible through virtual addresses
>which are translated by the MMU using the Page Tables. This question is
>similar to [2.]: If coreboot generates the Page Tables and the payload
>would start in PM as well (is this even possible? At least the Linux-Kernel
>has entry points for RM and PM) this would mean the payload needs to use
>the Page Tables generated by CB. That wouldn't be a problem as they're
>linked in the register CR3 anyways?
>
> And an unimportant bonus question:
>
>- Why does every modern CPU still start in RM? I do get the
>compatibility problem, but on the other hand: Do you need it for anything
>beside booting MS-DOS on your Ryzen? Is it really impossible for AMD and
>Intel to create a new CPU-generation with the x86-instruction set without
>RM, 16-bit-registers and 20-bit-mode registers like CS, SS etc. No m

Re: [coreboot] About Paging, Realmode and what is going on

[Philipp Stanner]

On 01.08.2017 16:49, Peter Stuge wrote:


Note that PM != paging. Neither coreboot nor proprietary BIOS
products used paging traditionally. Ron pushed for paging, there was
a bit of support. I don't know the current situation though.

Also note that PM != "flat real mode" or "32-bit real mode", it's not
really documented as a feature, so I don't know if it has an official
name.

What you do is prepare a GDT with most or all entries having base 0
size 4GB, then enable PM in CR0, ldgdt, set at least cs and ds
(usually more) to the 4GB GDT selectors, then disable PM in CR0 -
*and do not reload cs, ds and other selectors/segment registers*
once back in RM. This then allows 32-bit memory access in RM on 386 up.

Why would I want to address memory in RM with 32 Bits? I don't see any 
difference to using PM without Paging enabled.


--
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Trammell Hudson]

On Tue, Aug 01, 2017 at 02:49:27PM +, Peter Stuge wrote:
> Philipp Stanner wrote:
> [...]
> > * Why does every modern CPU still start in RM?
> 
> Many industries run on DOS. Many system developers have created
> in-house BIOS extensions. x86 will never fully lose its 16-bit legacy.

And, just to throw a wrench into things, the Bootguard startup ACM seems
to start in a 32-bit mode of some sort and then shift back to real mode
when it jumps into the legacy reset vector.  I'm not sure what the memory
map looks like during the ACM.

-- 
Trammell

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Peter Stuge]

Philipp Stanner wrote:
> the more I want to contribute and learn about low-level-code the less I 
> understand, it seems.

The x86 is a true rabbit hole. :)


>  2. When CB switches to PM - who generates and administrates the Page
> Tables and where?

Note that PM != paging. Neither coreboot nor proprietary BIOS
products used paging traditionally. Ron pushed for paging, there was
a bit of support. I don't know the current situation though.

Also note that PM != "flat real mode" or "32-bit real mode", it's not
really documented as a feature, so I don't know if it has an official
name.

What you do is prepare a GDT with most or all entries having base 0
size 4GB, then enable PM in CR0, ldgdt, set at least cs and ds
(usually more) to the 4GB GDT selectors, then disable PM in CR0 -
*and do not reload cs, ds and other selectors/segment registers*
once back in RM. This then allows 32-bit memory access in RM on 386 up.


>  4. Once CB is in PM it can't access physical addresses anymore? It
> doesn't need to, too?

That's a policy decision made in the GDT and the page tables. If
base=0 and paging disabled and/or page tables are set the right way,
then virtual == physical.


>  5. PM means RAM-access is only possible through virtual addresses
> which are translated by the MMU using the Page Tables.

If you stay in PM and enable paging, yes.


> If coreboot generates the Page Tables and the payload would start
> in PM as well (is this even possible? At least the Linux-Kernel has
> entry points for RM and PM) this would mean the payload needs to
> use the Page Tables generated by CB. That wouldn't be a problem as
> they're linked in the register CR3 anyways?

A few different bootloader binary interfaces exist on x86. There are
at least BIOS, UEFI, multiboot and coreboot. The general rule has
always been to not make very many assumptions, so expect every
bootloader and/or libpayload to set up their own flat real mode GDT.


> * Why does every modern CPU still start in RM?

Many industries run on DOS. Many system developers have created
in-house BIOS extensions. x86 will never fully lose its 16-bit legacy.


//Peter

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Patrick Georgi via coreboot]

2017-07-31 14:46 GMT+02:00 Nico Huber :
> No idea about the kernel's requirements
Windows 7's kernel uses int10 calls for checking out various graphics
properties, even when booting with UEFI (and therefore GOP drivers
better put some int10 handler in the right place if they want to
continue to support Win7).

> No idea, all of them? usually through a CSM (Compatibility Support
> Module). It's kind of the SeaBIOS of UEFI.
SeaBIOS can be compiled as CSM, too (though commercial UEFIs probably
don't ship with SeaBIOS for licensing reasons)


Patrick
-- 
Google Germany GmbH, ABC-Str. 19, 20354 Hamburg
Registergericht und -nummer: Hamburg, HRB 86891, Sitz der Gesellschaft: Hamburg
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Nico Huber]

On 31.07.2017 13:48, Philipp Stanner wrote:
> Well, many thanks.
> 
> I didn't expect it to work that way. Would be interesting to know what
> Windows 7 needs BIOS calls for.

It seems to be their policy to have a boot loader (and probably kernel)
that can't work on its own. So Windows (including current versions) al-
ways need either BIOS calls or UEFI services. No idea about the kernel's
requirements, but the bootloader part could probably be fixed easily by
implementing Multiboot ;)

> 
> Any idea of traditional IBM-PC-Bios-calls are still available in
> UEFI-Firmware?

No idea, all of them? usually through a CSM (Compatibility Support
Module). It's kind of the SeaBIOS of UEFI.

Nico

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Philipp Stanner]

Well, many thanks.

I didn't expect it to work that way. Would be interesting to know what 
Windows 7 needs BIOS calls for.


Any idea of traditional IBM-PC-Bios-calls are still available in 
UEFI-Firmware?



On 31.07.2017 11:43, Patrick Georgi via coreboot wrote:

2017-07-31 10:52 GMT+02:00 Philipp Stanner :

1. cb switches the CPU immediately to Protected Mode, yet Payloads like seaBIOS 
work in Real Mode. Does coreboot switch the CPU always back to RM before 
jumping to the payload?

No, payloads are started in pmode.


2. When CB switches to PM - who generates and administrates the Page Tables and 
where?

We use flat mode: 4gb segments starting at 0, for code and data.
Virtual address == logical address == physical address


3. Gustavo Duarte writes that GRUB switches from protected mode to real mode and 
vice versa all the time to address >1MiB of RAM and also use the BIOS-calls. If 
this is true using GRUB as payload would not work, as GRUB needs to call the 
non-existent BIOS, right?

It uses BIOS calls, except when built for coreboot.


4. Once CB is in PM it can't access physical addresses anymore? It doesn't need 
to, too?

We use flat mode, see above.


5. PM means RAM-access is only possible through virtual addresses which are 
translated by the MMU using the Page Tables. This question is similar to [2.]: 
If coreboot generates the Page Tables and the payload would start in PM as well 
(is this even possible? At least the Linux-Kernel has entry points for RM and 
PM) this would mean the payload needs to use the Page Tables generated by CB. 
That wouldn't be a problem as they're linked in the register CR3 anyways?

As stated above, payloads start in pmode. As stated above, we use a
flat representation which comes with no surprises. The payload can
then reconfigure the system to setup its own configuration.


Why does every modern CPU still start in RM? I do get the compatibility 
problem, but on the other hand: Do you need it for anything beside booting 
MS-DOS on your Ryzen? Is it really impossible for AMD and Intel to create a new 
CPU-generation with the x86-instruction set without RM, 16-bit-registers and 
20-bit-mode registers like CS, SS etc. No modern OS uses bios calls. No CPU is 
ever switched to RM again after booting up. They should get rid of this old 
stuff.

"Every modern x86 CPU". In the end, that's something to ask the CPU
vendors (but don't expect any answers). Some guesses:

1. Windows 7 uses BIOS calls (although they stopped switching back to
real mode for that, they use x86 16bit emulation. still, BIOS services
need to be there)
2. CPUs might not be switched back to real mode today, but from 32bit
modes it's a pretty short route to vm86 modes, which are effectively
identical to real mode and still in use.
3. Why change a working system and risk compatibility issues? x86's
biggest selling point is compatibility, and if you forfeit that, users
may move off your architecture entirely.


Regards,
Patrick



--
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

Re: [coreboot] About Paging, Realmode and what is going on

[Patrick Georgi via coreboot]

2017-07-31 10:52 GMT+02:00 Philipp Stanner :
> 1. cb switches the CPU immediately to Protected Mode, yet Payloads like 
> seaBIOS work in Real Mode. Does coreboot switch the CPU always back to RM 
> before jumping to the payload?
No, payloads are started in pmode.

> 2. When CB switches to PM - who generates and administrates the Page Tables 
> and where?
We use flat mode: 4gb segments starting at 0, for code and data.
Virtual address == logical address == physical address

> 3. Gustavo Duarte writes that GRUB switches from protected mode to real mode 
> and vice versa all the time to address >1MiB of RAM and also use the 
> BIOS-calls. If this is true using GRUB as payload would not work, as GRUB 
> needs to call the non-existent BIOS, right?
It uses BIOS calls, except when built for coreboot.

> 4. Once CB is in PM it can't access physical addresses anymore? It doesn't 
> need to, too?
We use flat mode, see above.

> 5. PM means RAM-access is only possible through virtual addresses which are 
> translated by the MMU using the Page Tables. This question is similar to 
> [2.]: If coreboot generates the Page Tables and the payload would start in PM 
> as well (is this even possible? At least the Linux-Kernel has entry points 
> for RM and PM) this would mean the payload needs to use the Page Tables 
> generated by CB. That wouldn't be a problem as they're linked in the register 
> CR3 anyways?
As stated above, payloads start in pmode. As stated above, we use a
flat representation which comes with no surprises. The payload can
then reconfigure the system to setup its own configuration.

> Why does every modern CPU still start in RM? I do get the compatibility 
> problem, but on the other hand: Do you need it for anything beside booting 
> MS-DOS on your Ryzen? Is it really impossible for AMD and Intel to create a 
> new CPU-generation with the x86-instruction set without RM, 16-bit-registers 
> and 20-bit-mode registers like CS, SS etc. No modern OS uses bios calls. No 
> CPU is ever switched to RM again after booting up. They should get rid of 
> this old stuff.
"Every modern x86 CPU". In the end, that's something to ask the CPU
vendors (but don't expect any answers). Some guesses:

1. Windows 7 uses BIOS calls (although they stopped switching back to
real mode for that, they use x86 16bit emulation. still, BIOS services
need to be there)
2. CPUs might not be switched back to real mode today, but from 32bit
modes it's a pretty short route to vm86 modes, which are effectively
identical to real mode and still in use.
3. Why change a working system and risk compatibility issues? x86's
biggest selling point is compatibility, and if you forfeit that, users
may move off your architecture entirely.


Regards,
Patrick
-- 
Google Germany GmbH, ABC-Str. 19, 20354 Hamburg
Registergericht und -nummer: Hamburg, HRB 86891, Sitz der Gesellschaft: Hamburg
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot

[coreboot] About Paging, Realmode and what is going on

[Philipp Stanner]

Dear folks and techpriests,

the more I want to contribute and learn about low-level-code the less I 
understand, it seems.


1. cb switches the CPU immediately to Protected Mode, yet Payloads like
   seaBIOS work in Real Mode. Does coreboot switch the CPU always back
   to RM before jumping to the payload?
2. When CB switches to PM - who generates and administrates the Page
   Tables and where?
3. Gustavo Duarte writes
    that
   GRUB switches from protected mode to real mode and vice versa all
   the time to address >1MiB of RAM and also use the BIOS-calls. If
   this is true using GRUB as payload would not work, as GRUB needs to
   call the non-existent BIOS, right?
4. Once CB is in PM it can't access physical addresses anymore? It
   doesn't need to, too?
5. PM means RAM-access is only possible through virtual addresses which
   are translated by the MMU using the Page Tables. This question is
   similar to [2.]: If coreboot generates the Page Tables and the
   payload would start in PM as well (is this even possible? At least
   the Linux-Kernel has entry points for RM and PM) this would mean the
   payload needs to use the Page Tables generated by CB. That wouldn't
   be a problem as they're linked in the register CR3 anyways?

And an unimportant bonus question:

 * Why does every modern CPU still start in RM? I do get the
   compatibility problem, but on the other hand: Do you need it for
   anything beside booting MS-DOS on your Ryzen? Is it really
   impossible for AMD and Intel to create a new CPU-generation with the
   x86-instruction set without RM, 16-bit-registers and 20-bit-mode
   registers like CS, SS etc. No modern OS uses bios calls. No CPU is
   ever switched to RM again after booting up. They should get rid of
   this old stuff.

Would be cool if someone could put this in its true light.

Thanks,

Philipp

-- 
coreboot mailing list: coreboot@coreboot.org
https://mail.coreboot.org/mailman/listinfo/coreboot