Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-29 Thread Jeremy Fitzhardinge
Eric W. Biederman wrote:
> Any chance you had CONFIG_DEBUG_PAGEALLOC selected?
>   

Yes, that's quite likely.

J
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-29 Thread Eric W. Biederman
Jeremy Fitzhardinge <[EMAIL PROTECTED]> writes:

> Eric W. Biederman wrote:
>> And it just occurred to me PSE disabled, otherwise you would not have
>> needed more than 4 pages.  I supposed you were testing the Xen case.
>>   
>
> No, actually, I wasn't.  It was booting native (the Xen boot path
> doesn't go that way), and it should have booted like a normal native
> kernel despite having Xen compiled in.  Hm, not sure why it wouldn't be
> using the PSE path.

Any chance you had CONFIG_DEBUG_PAGEALLOC selected?

Apparently that also disabled PSE...

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-29 Thread Eric W. Biederman
Jeremy Fitzhardinge [EMAIL PROTECTED] writes:

 Eric W. Biederman wrote:
 And it just occurred to me PSE disabled, otherwise you would not have
 needed more than 4 pages.  I supposed you were testing the Xen case.
   

 No, actually, I wasn't.  It was booting native (the Xen boot path
 doesn't go that way), and it should have booted like a normal native
 kernel despite having Xen compiled in.  Hm, not sure why it wouldn't be
 using the PSE path.

Any chance you had CONFIG_DEBUG_PAGEALLOC selected?

Apparently that also disabled PSE...

Eric
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-29 Thread Jeremy Fitzhardinge
Eric W. Biederman wrote:
 Any chance you had CONFIG_DEBUG_PAGEALLOC selected?
   

Yes, that's quite likely.

J
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Jeremy Fitzhardinge
Chris Wright wrote:
> I was using real hardware with your .config when I reproduced it.
>   

Yes, I first found it on real hardware. I haven't tested my fix on real
hardware yet, but it seems OK on kvm.

J
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Chris Wright
* Jeremy Fitzhardinge ([EMAIL PROTECTED]) wrote:
> Eric W. Biederman wrote:
> > Then why you had to allocate enough pages to cause a failure has me stumped.
> > Perhaps there is some other bug?
> 
> Perhaps, but nothing comes to mind. I'll see what happens when I boot
> this kernel on real hardware (rather than kvm).

I was using real hardware with your .config when I reproduced it.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Jeremy Fitzhardinge
Eric W. Biederman wrote:
> Then why you had to allocate enough pages to cause a failure has me stumped.
> Perhaps there is some other bug?
>   

Perhaps, but nothing comes to mind. I'll see what happens when I boot
this kernel on real hardware (rather than kvm).

J
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Eric W. Biederman
Jeremy Fitzhardinge <[EMAIL PROTECTED]> writes:

> Eric W. Biederman wrote:
>> Since you have PSE disabled for Xen my hunch is that somehow that
>> got left on for your test boot.
>
> No.  Under Xen cpuid masks out PSE (and complains if you try to set it
> in a pte), but when booting native it will just use a plain unadorned cpuid.

Then why you had to allocate enough pages to cause a failure has me stumped.
Perhaps there is some other bug?

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Jeremy Fitzhardinge
Eric W. Biederman wrote:
> Since you have PSE disabled for Xen my hunch is that somehow that
> got left on for your test boot.

No.  Under Xen cpuid masks out PSE (and complains if you try to set it
in a pte), but when booting native it will just use a plain unadorned cpuid.

J
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Eric W. Biederman
Jeremy Fitzhardinge <[EMAIL PROTECTED]> writes:

> Eric W. Biederman wrote:
>> And it just occurred to me PSE disabled, otherwise you would not have
>> needed more than 4 pages.  I supposed you were testing the Xen case.
>>   

>
> No, actually, I wasn't.  It was booting native (the Xen boot path
> doesn't go that way), and it should have booted like a normal native
> kernel despite having Xen compiled in.  Hm, not sure why it wouldn't be
> using the PSE path.

Since you have PSE disabled for Xen my hunch is that somehow that
got left on for your test boot.

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Jeremy Fitzhardinge
Eric W. Biederman wrote:
> And it just occurred to me PSE disabled, otherwise you would not have
> needed more than 4 pages.  I supposed you were testing the Xen case.
>   

No, actually, I wasn't.  It was booting native (the Xen boot path
doesn't go that way), and it should have booted like a normal native
kernel despite having Xen compiled in.  Hm, not sure why it wouldn't be
using the PSE path.

J
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Eric W. Biederman
Jeremy Fitzhardinge <[EMAIL PROTECTED]> writes:

> Eric W. Biederman wrote:
>> Jeremy did your kernel have PAE enabled?
>>
>> It just occurred to me that we have at all of the memory below 1M (say about
>> 512K) mapped and available to setup new mappings.
>>
>> The only way I can see a page fault happening is if you were using a PAE
>> enabled kernel (so you were not updating the current page tables) and
>> you have more than 256M of low memory, and we don't get any much extra
>> from always mapping 4M at a time.
>
> Yes, that's the situation.  PAE enabled, ~768MB of memory and a large
> kernel which mostly fills the 8M mapping.

And it just occurred to me PSE disabled, otherwise you would not have
needed more than 4 pages.  I supposed you were testing the Xen case.

That said I know for certain now why we have never seen this in
production because all cpus that fit this configuration also support
PSE.

So the way I think we should properly fix this is:
- Enable PAE early so we are updating our current page table.
- Pass a limit to the bootmem allocator so we do not get pages
  pass the point where we have setup page tables.
- Handle memory allocation failures.

However only the first part is necessary to solve the problem,
in practice even if you are using 4K pages.

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Jeremy Fitzhardinge
Eric W. Biederman wrote:
> Jeremy did your kernel have PAE enabled?
>
> It just occurred to me that we have at all of the memory below 1M (say about
> 512K) mapped and available to setup new mappings.
>
> The only way I can see a page fault happening is if you were using a PAE
> enabled kernel (so you were not updating the current page tables) and
> you have more than 256M of low memory, and we don't get any much extra
> from always mapping 4M at a time.

Yes, that's the situation.  PAE enabled, ~768MB of memory and a large
kernel which mostly fills the 8M mapping.

J
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Eric W. Biederman
Jeremy Fitzhardinge <[EMAIL PROTECTED]> writes:

> Chuck Ebbert wrote:
>> H. Peter Anvin wrote:
>>   
>>> Andi Kleen wrote:
>>> 
 Then we would have seen reports surely?
   
>
> Yes, I would have thought so.  It surprised me that such an obvious bug
> could be there, apparently for a long time.  But it's real, and
> potentially affects everyone.  It probably doesn't affect highly modular
> distros much, since the kernel itself will be relatively small.
>
>> I never saw a description of the symptoms of encountering this bug.
>> Does it just hang, or what?
>>   
>
> You get an early-fault message on-screen, assuming that's enabled;
> otherwise it will just appear to hang.  It happens in pagetable_init,
> when it allocates a new pagetable above the head.S mapping (8M in my
> case).  It will only hit if the kernel size approaches a 4M boundary,
> since it won't leave enough space mapped to construct the lowmem mappings.
>
> It only affects native booting, since under Xen all those mappings have
> already been constructed.  It happened to me with a paravirt kernel that
> happened to Xen compiled into it, but that was irrelevent (though
> misleading; the 40k difference in kernel size was enough to make it not
> happen in a non-Xen kernel).

Jeremy did your kernel have PAE enabled?

It just occurred to me that we have at all of the memory below 1M (say about
512K) mapped and available to setup new mappings.

The only way I can see a page fault happening is if you were using a PAE
enabled kernel (so you were not updating the current page tables) and
you have more than 256M of low memory, and we don't get any much extra
from always mapping 4M at a time.

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Eric W. Biederman
Jeremy Fitzhardinge [EMAIL PROTECTED] writes:

 Chuck Ebbert wrote:
 H. Peter Anvin wrote:
   
 Andi Kleen wrote:
 
 Then we would have seen reports surely?
   

 Yes, I would have thought so.  It surprised me that such an obvious bug
 could be there, apparently for a long time.  But it's real, and
 potentially affects everyone.  It probably doesn't affect highly modular
 distros much, since the kernel itself will be relatively small.

 I never saw a description of the symptoms of encountering this bug.
 Does it just hang, or what?
   

 You get an early-fault message on-screen, assuming that's enabled;
 otherwise it will just appear to hang.  It happens in pagetable_init,
 when it allocates a new pagetable above the head.S mapping (8M in my
 case).  It will only hit if the kernel size approaches a 4M boundary,
 since it won't leave enough space mapped to construct the lowmem mappings.

 It only affects native booting, since under Xen all those mappings have
 already been constructed.  It happened to me with a paravirt kernel that
 happened to Xen compiled into it, but that was irrelevent (though
 misleading; the 40k difference in kernel size was enough to make it not
 happen in a non-Xen kernel).

Jeremy did your kernel have PAE enabled?

It just occurred to me that we have at all of the memory below 1M (say about
512K) mapped and available to setup new mappings.

The only way I can see a page fault happening is if you were using a PAE
enabled kernel (so you were not updating the current page tables) and
you have more than 256M of low memory, and we don't get any much extra
from always mapping 4M at a time.

Eric
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Jeremy Fitzhardinge
Eric W. Biederman wrote:
 Jeremy did your kernel have PAE enabled?

 It just occurred to me that we have at all of the memory below 1M (say about
 512K) mapped and available to setup new mappings.

 The only way I can see a page fault happening is if you were using a PAE
 enabled kernel (so you were not updating the current page tables) and
 you have more than 256M of low memory, and we don't get any much extra
 from always mapping 4M at a time.

Yes, that's the situation.  PAE enabled, ~768MB of memory and a large
kernel which mostly fills the 8M mapping.

J
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Eric W. Biederman
Jeremy Fitzhardinge [EMAIL PROTECTED] writes:

 Eric W. Biederman wrote:
 Jeremy did your kernel have PAE enabled?

 It just occurred to me that we have at all of the memory below 1M (say about
 512K) mapped and available to setup new mappings.

 The only way I can see a page fault happening is if you were using a PAE
 enabled kernel (so you were not updating the current page tables) and
 you have more than 256M of low memory, and we don't get any much extra
 from always mapping 4M at a time.

 Yes, that's the situation.  PAE enabled, ~768MB of memory and a large
 kernel which mostly fills the 8M mapping.

And it just occurred to me PSE disabled, otherwise you would not have
needed more than 4 pages.  I supposed you were testing the Xen case.

That said I know for certain now why we have never seen this in
production because all cpus that fit this configuration also support
PSE.

So the way I think we should properly fix this is:
- Enable PAE early so we are updating our current page table.
- Pass a limit to the bootmem allocator so we do not get pages
  pass the point where we have setup page tables.
- Handle memory allocation failures.

However only the first part is necessary to solve the problem,
in practice even if you are using 4K pages.

Eric
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Jeremy Fitzhardinge
Eric W. Biederman wrote:
 And it just occurred to me PSE disabled, otherwise you would not have
 needed more than 4 pages.  I supposed you were testing the Xen case.
   

No, actually, I wasn't.  It was booting native (the Xen boot path
doesn't go that way), and it should have booted like a normal native
kernel despite having Xen compiled in.  Hm, not sure why it wouldn't be
using the PSE path.

J
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Eric W. Biederman
Jeremy Fitzhardinge [EMAIL PROTECTED] writes:

 Eric W. Biederman wrote:
 And it just occurred to me PSE disabled, otherwise you would not have
 needed more than 4 pages.  I supposed you were testing the Xen case.
   


 No, actually, I wasn't.  It was booting native (the Xen boot path
 doesn't go that way), and it should have booted like a normal native
 kernel despite having Xen compiled in.  Hm, not sure why it wouldn't be
 using the PSE path.

Since you have PSE disabled for Xen my hunch is that somehow that
got left on for your test boot.

Eric
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Jeremy Fitzhardinge
Eric W. Biederman wrote:
 Since you have PSE disabled for Xen my hunch is that somehow that
 got left on for your test boot.

No.  Under Xen cpuid masks out PSE (and complains if you try to set it
in a pte), but when booting native it will just use a plain unadorned cpuid.

J
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Eric W. Biederman
Jeremy Fitzhardinge [EMAIL PROTECTED] writes:

 Eric W. Biederman wrote:
 Since you have PSE disabled for Xen my hunch is that somehow that
 got left on for your test boot.

 No.  Under Xen cpuid masks out PSE (and complains if you try to set it
 in a pte), but when booting native it will just use a plain unadorned cpuid.

Then why you had to allocate enough pages to cause a failure has me stumped.
Perhaps there is some other bug?

Eric
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Jeremy Fitzhardinge
Eric W. Biederman wrote:
 Then why you had to allocate enough pages to cause a failure has me stumped.
 Perhaps there is some other bug?
   

Perhaps, but nothing comes to mind. I'll see what happens when I boot
this kernel on real hardware (rather than kvm).

J
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Chris Wright
* Jeremy Fitzhardinge ([EMAIL PROTECTED]) wrote:
 Eric W. Biederman wrote:
  Then why you had to allocate enough pages to cause a failure has me stumped.
  Perhaps there is some other bug?
 
 Perhaps, but nothing comes to mind. I'll see what happens when I boot
 this kernel on real hardware (rather than kvm).

I was using real hardware with your .config when I reproduced it.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-25 Thread Jeremy Fitzhardinge
Chris Wright wrote:
 I was using real hardware with your .config when I reproduced it.
   

Yes, I first found it on real hardware. I haven't tested my fix on real
hardware yet, but it seems OK on kvm.

J
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread H. Peter Anvin

Eric W. Biederman wrote:

Jeremy Fitzhardinge <[EMAIL PROTECTED]> writes:


H. Peter Anvin wrote:

It would be *trivial* to make a certain number of page table slots
available at the end of the head.S-generated map. 

Or you could use a fixmap.


That certain number of page table slots should be the fixmap slots.
If you do that bt_ioremap works, we remove a boatload of special cases
and the code becomes simpler.


It's probably worth noting that several of the non-PC ports already do 
exactly this, although in a hacky way.  Grep for pg0 and you'll find it.


-hpa
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Eric W. Biederman
Jeremy Fitzhardinge <[EMAIL PROTECTED]> writes:

> H. Peter Anvin wrote:
>> It would be *trivial* to make a certain number of page table slots
>> available at the end of the head.S-generated map. 
>
> Or you could use a fixmap.

That certain number of page table slots should be the fixmap slots.
If you do that bt_ioremap works, we remove a boatload of special cases
and the code becomes simpler.

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Jeremy Fitzhardinge
H. Peter Anvin wrote:
> It would be *trivial* to make a certain number of page table slots
> available at the end of the head.S-generated map. 

Or you could use a fixmap.

J
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread H. Peter Anvin

Jeremy Fitzhardinge wrote:


Why not?  Er, except in the case where the page is needed to map itself
- but that can be dealt with with a transient fixmap mapping.



It would be *trivial* to make a certain number of page table slots 
available at the end of the head.S-generated map.


-hpa
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Jeremy Fitzhardinge
Eric W. Biederman wrote:
> If (cpu_has_pse) it may only be an additional two pages.
> INIT_MAP_BEYOND is currently mapping a lot more then that.
>   

Ah, yes.  It allocates an extra two pages for pagetables, and then maps
an extra 8MB or so.

>> Would that be necessary?  Is there any need to remap it?  Couldn't you
>> just do the straightforward phys->virt mapping and use the page where it
>> lies?
>> 
>
> Catch-22.  If the page is not mapped yet you can't use it where it lies.

Why not?  Er, except in the case where the page is needed to map itself
- but that can be dealt with with a transient fixmap mapping.

J
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Eric W. Biederman
Jeremy Fitzhardinge <[EMAIL PROTECTED]> writes:

> Eric W. Biederman wrote:
>> Consider a memory hole of size 8M immediately after our bootmem bitmap.
>> head.S which knows nothing of holes will map the pages of the hole
>> into the initial page tables assuming that is where the page tables
>> will live.
>>   
>
> Sure, but considering we're only talking about mapping an additional *2*
> pages, the chances are that it will affect a kernel as much as the early
> head.S mapping.  It's well within the variability of fairly similar
> kernel configs.

If (cpu_has_pse) it may only be an additional two pages.
INIT_MAP_BEYOND is currently mapping a lot more then that.

>> As for how to fix this, we need to call boot_ioremap or better
>> bt_ioremap on the address returned from alloc_bootmem_pages_low to
>> force the allocated page into the page table.  After we have setup
>> the page table page we can call unmap the page to free up the scarce
>> early mapping resource.
>>   
>
> Would that be necessary?  Is there any need to remap it?  Couldn't you
> just do the straightforward phys->virt mapping and use the page where it
> lies?

Catch-22.  If the page is not mapped yet you can't use it where it lies.

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Jeremy Fitzhardinge
Eric W. Biederman wrote:
> Consider a memory hole of size 8M immediately after our bootmem bitmap.
> head.S which knows nothing of holes will map the pages of the hole
> into the initial page tables assuming that is where the page tables
> will live.
>   

Sure, but considering we're only talking about mapping an additional *2*
pages, the chances are that it will affect a kernel as much as the early
head.S mapping.  It's well within the variability of fairly similar
kernel configs.

> As for how to fix this, we need to call boot_ioremap or better
> bt_ioremap on the address returned from alloc_bootmem_pages_low to
> force the allocated page into the page table.  After we have setup
> the page table page we can call unmap the page to free up the scarce
> early mapping resource.
>   

Would that be necessary?  Is there any need to remap it?  Couldn't you
just do the straightforward phys->virt mapping and use the page where it
lies?

J
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Eric W. Biederman
"H. Peter Anvin" <[EMAIL PROTECTED]> writes:

> Agreed.  However, saying that your patch shouldn't go into the mainline kernel
> until that has been fixed is spurious and wrong.  It fixes a real problem with
> minimal risk.

For a stable and frozen kernel it is probably the best we can do.

However the code is based on confusion of what should be happening and
it increases the complexity of the code and pushes us towards the edge of
code that is so crufty it is impossible to maintain.

We need to simplify this mess not make it more convoluted.

Did anyone else notice that if we don't have PAE we continue to use
the initial page tables 4K page mappings of the kernel?

This probably explains why experiments to 4MB align the i386 kernel
to improve performance failed.

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Andi Kleen
On Monday 23 April 2007 19:45:41 H. Peter Anvin wrote:
> Eric W. Biederman wrote:
> > 
> > - I know of one system that had BIOS tables at 16MB I believe (and
> >   thus had a fairly low hole).
> > 
> 
> Please name names, otherwise this is just rumouring.  Seriously.  We 
> have enough cargo-cult programming as it is.

Unisys did this at some point in their large machines, but they fixed
that in a later BIOS.

Anyways, boot up should be robust against any holes if possible.

-Andi

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Eric W. Biederman
"H. Peter Anvin" <[EMAIL PROTECTED]> writes:

> Eric W. Biederman wrote:
>>
>> - I know of one system that had BIOS tables at 16MB I believe (and
>>   thus had a fairly low hole).
>>
>
> Please name names, otherwise this is just rumouring.  Seriously.  We have 
> enough
> cargo-cult programming as it is.

The unisys systems.  I don't remember exactly where they put there ACPI
tables but it was low enough that the default address for the kdump kernels
conflicted with it.  So this is likely 16MB or 32MB

> A lot of old ISA systems had an option to put a memory hole at 15-16 MB to
> support ISA video cards with linear frame buffer.  It very rarely got used,
> since VLB took over around the same time.

No this was a much larger and more modern system.

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread H. Peter Anvin

Eric W. Biederman wrote:


- I know of one system that had BIOS tables at 16MB I believe (and
  thus had a fairly low hole).



Please name names, otherwise this is just rumouring.  Seriously.  We 
have enough cargo-cult programming as it is.


A lot of old ISA systems had an option to put a memory hole at 15-16 MB 
to support ISA video cards with linear frame buffer.  It very rarely got 
used, since VLB took over around the same time.


-hpa
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Eric W. Biederman
Jeremy Fitzhardinge <[EMAIL PROTECTED]> writes:

> H. Peter Anvin wrote:
>> Since we allocate the maximum possible memory statically, I fail to
>> see how holes could make the situation any worse, or better. 
>
> No, we map enough space to map 4G (~4 pages), but we don't actually map
> 4G.  If a hole happened to start within that 4 page mapping, then the
> memory still wouldn't be available for allocation.
>
> I think this is a bit of a spurious argument though, since if it were
> really a problem we'd have to worry about holes hitting the kernel image
> too.  As far as I can see, that's not considered to be a problem.

- holes hitting the kernel image is not something we can do anything about.

- I have always believed we need to export enough information so the
  bootloader can verify that we don't hit a hole in the initial
  kernel image.

- I know of one system that had BIOS tables at 16MB I believe (and
  thus had a fairly low hole).

- Given that we are actually increasing (not decreasing) the number of
  scenarios where we boot the kernel it probably makes sense to be
  as robust as we can.

- If we support putting the ramdisk immediately after the kernel image
  in memory we will actually hit this case in practice. 

> I think the real point is that there's currently a subtle dependency
> between head.S and bootmem allocation which happens between start_kernel
> and pagetable_init.  Your patch preventing over-mapping should make them
> easier to smoke out as it currently stands, but eliminating the problem
> by making alloc_bootmem create the mappings for itself does have
> appeal.  There would still be the dependency on head.S to map the kernel
> itself and the bootmem allocator bitmap.

Agreed.  However that is essentially all statically allocated memory
and the best we can do at this point in time.

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Eric W. Biederman
"H. Peter Anvin" <[EMAIL PROTECTED]> writes:

> Since we allocate the maximum possible memory statically, I fail to see how
> holes could make the situation any worse, or better.

Consider a memory hole of size 8M immediately after our bootmem bitmap.
head.S which knows nothing of holes will map the pages of the hole
into the initial page tables assuming that is where the page tables
will live.

alloc_bootmem_low_pages which understands holes will skip those pages
and return a higher address, which is outside of our initial mapping
of pages.

As for how to fix this, we need to call boot_ioremap or better
bt_ioremap on the address returned from alloc_bootmem_pages_low to
force the allocated page into the page table.  After we have setup
the page table page we can call unmap the page to free up the scarce
early mapping resource.

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread H. Peter Anvin

Jeremy Fitzhardinge wrote:

H. Peter Anvin wrote:

Since we allocate the maximum possible memory statically, I fail to
see how holes could make the situation any worse, or better. 


No, we map enough space to map 4G (~4 pages), but we don't actually map
4G.  If a hole happened to start within that 4 page mapping, then the
memory still wouldn't be available for allocation.

I think this is a bit of a spurious argument though, since if it were
really a problem we'd have to worry about holes hitting the kernel image
too.  As far as I can see, that's not considered to be a problem.

I think the real point is that there's currently a subtle dependency
between head.S and bootmem allocation which happens between start_kernel
and pagetable_init.  Your patch preventing over-mapping should make them
easier to smoke out as it currently stands, but eliminating the problem
by making alloc_bootmem create the mappings for itself does have
appeal.  There would still be the dependency on head.S to map the kernel
itself and the bootmem allocator bitmap.



Agreed.  However, saying that your patch shouldn't go into the mainline 
kernel until that has been fixed is spurious and wrong.  It fixes a real 
problem with minimal risk.


-hpa
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Jeremy Fitzhardinge
H. Peter Anvin wrote:
> Since we allocate the maximum possible memory statically, I fail to
> see how holes could make the situation any worse, or better. 

No, we map enough space to map 4G (~4 pages), but we don't actually map
4G.  If a hole happened to start within that 4 page mapping, then the
memory still wouldn't be available for allocation.

I think this is a bit of a spurious argument though, since if it were
really a problem we'd have to worry about holes hitting the kernel image
too.  As far as I can see, that's not considered to be a problem.

I think the real point is that there's currently a subtle dependency
between head.S and bootmem allocation which happens between start_kernel
and pagetable_init.  Your patch preventing over-mapping should make them
easier to smoke out as it currently stands, but eliminating the problem
by making alloc_bootmem create the mappings for itself does have
appeal.  There would still be the dependency on head.S to map the kernel
itself and the bootmem allocator bitmap.

J
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread H. Peter Anvin

Jeremy Fitzhardinge wrote:

Eric W. Biederman wrote:

The only way to ensure this will not happen is to do what we do
on x86_64 and map the new page table page into our address space
before we write to it.  Assuming the page we allocate is already
mapped is simply not robust.
  


So you mean make alloc_bootmem make sure there's a mapping for the
returned page?  That would be simple enough, though it might interact
strangely with the actual construction of the memory mappings (what if
the page alloc_bootmem just allocated for the pagetable is the page
pagetable_init is actually trying to map?).

I'm not sure about the issue with holes.  I assume you mean that just
because head.S maps a chunk of memory, it isn't necessarily available
for allocation because its a hole.  But does alloc_bootmem know to avoid
them anyway?  Has it already parsed e820 at that point?



Since we allocate the maximum possible memory statically, I fail to see 
how holes could make the situation any worse, or better.


-hpa
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Jeremy Fitzhardinge
Eric W. Biederman wrote:
> The only way to ensure this will not happen is to do what we do
> on x86_64 and map the new page table page into our address space
> before we write to it.  Assuming the page we allocate is already
> mapped is simply not robust.
>   

So you mean make alloc_bootmem make sure there's a mapping for the
returned page?  That would be simple enough, though it might interact
strangely with the actual construction of the memory mappings (what if
the page alloc_bootmem just allocated for the pagetable is the page
pagetable_init is actually trying to map?).

I'm not sure about the issue with holes.  I assume you mean that just
because head.S maps a chunk of memory, it isn't necessarily available
for allocation because its a hole.  But does alloc_bootmem know to avoid
them anyway?  Has it already parsed e820 at that point?

J
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread H. Peter Anvin

Eric W. Biederman wrote:


I happened to be looking at this stretch of code and I have realized
that this is quite simply the wrong fix.

The problem is that it depends intimately on the details of
alloc_bootmem_pages_low.  Essentially the problem is that when
we are setting up the identity mappings in paging_init we assume
the identity mappings already exist.

If there are holes in the memory map or someone changes the way
pages are returned from alloc_bootmem_pages_low() this code
will break again.



How would holes in the memory map affect it?

-hpa
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Eric W. Biederman
Jeremy Fitzhardinge <[EMAIL PROTECTED]> writes:

> Chuck Ebbert wrote:
>> H. Peter Anvin wrote:
>>   
>>> Andi Kleen wrote:
>>> 
 Then we would have seen reports surely?
   
>
> Yes, I would have thought so.  It surprised me that such an obvious bug
> could be there, apparently for a long time.  But it's real, and
> potentially affects everyone.  It probably doesn't affect highly modular
> distros much, since the kernel itself will be relatively small.
>
>> I never saw a description of the symptoms of encountering this bug.
>> Does it just hang, or what?
>>   
>
> You get an early-fault message on-screen, assuming that's enabled;
> otherwise it will just appear to hang.  It happens in pagetable_init,
> when it allocates a new pagetable above the head.S mapping (8M in my
> case).  It will only hit if the kernel size approaches a 4M boundary,
> since it won't leave enough space mapped to construct the lowmem mappings.
>
> It only affects native booting, since under Xen all those mappings have
> already been constructed.  It happened to me with a paravirt kernel that
> happened to Xen compiled into it, but that was irrelevent (though
> misleading; the 40k difference in kernel size was enough to make it not
> happen in a non-Xen kernel).

I happened to be looking at this stretch of code and I have realized
that this is quite simply the wrong fix.

The problem is that it depends intimately on the details of
alloc_bootmem_pages_low.  Essentially the problem is that when
we are setting up the identity mappings in paging_init we assume
the identity mappings already exist.

If there are holes in the memory map or someone changes the way
pages are returned from alloc_bootmem_pages_low() this code
will break again.

The only way to ensure this will not happen is to do what we do
on x86_64 and map the new page table page into our address space
before we write to it.  Assuming the page we allocate is already
mapped is simply not robust.

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Eric W. Biederman
Jeremy Fitzhardinge [EMAIL PROTECTED] writes:

 Chuck Ebbert wrote:
 H. Peter Anvin wrote:
   
 Andi Kleen wrote:
 
 Then we would have seen reports surely?
   

 Yes, I would have thought so.  It surprised me that such an obvious bug
 could be there, apparently for a long time.  But it's real, and
 potentially affects everyone.  It probably doesn't affect highly modular
 distros much, since the kernel itself will be relatively small.

 I never saw a description of the symptoms of encountering this bug.
 Does it just hang, or what?
   

 You get an early-fault message on-screen, assuming that's enabled;
 otherwise it will just appear to hang.  It happens in pagetable_init,
 when it allocates a new pagetable above the head.S mapping (8M in my
 case).  It will only hit if the kernel size approaches a 4M boundary,
 since it won't leave enough space mapped to construct the lowmem mappings.

 It only affects native booting, since under Xen all those mappings have
 already been constructed.  It happened to me with a paravirt kernel that
 happened to Xen compiled into it, but that was irrelevent (though
 misleading; the 40k difference in kernel size was enough to make it not
 happen in a non-Xen kernel).

I happened to be looking at this stretch of code and I have realized
that this is quite simply the wrong fix.

The problem is that it depends intimately on the details of
alloc_bootmem_pages_low.  Essentially the problem is that when
we are setting up the identity mappings in paging_init we assume
the identity mappings already exist.

If there are holes in the memory map or someone changes the way
pages are returned from alloc_bootmem_pages_low() this code
will break again.

The only way to ensure this will not happen is to do what we do
on x86_64 and map the new page table page into our address space
before we write to it.  Assuming the page we allocate is already
mapped is simply not robust.

Eric
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread H. Peter Anvin

Eric W. Biederman wrote:


I happened to be looking at this stretch of code and I have realized
that this is quite simply the wrong fix.

The problem is that it depends intimately on the details of
alloc_bootmem_pages_low.  Essentially the problem is that when
we are setting up the identity mappings in paging_init we assume
the identity mappings already exist.

If there are holes in the memory map or someone changes the way
pages are returned from alloc_bootmem_pages_low() this code
will break again.



How would holes in the memory map affect it?

-hpa
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Jeremy Fitzhardinge
Eric W. Biederman wrote:
 The only way to ensure this will not happen is to do what we do
 on x86_64 and map the new page table page into our address space
 before we write to it.  Assuming the page we allocate is already
 mapped is simply not robust.
   

So you mean make alloc_bootmem make sure there's a mapping for the
returned page?  That would be simple enough, though it might interact
strangely with the actual construction of the memory mappings (what if
the page alloc_bootmem just allocated for the pagetable is the page
pagetable_init is actually trying to map?).

I'm not sure about the issue with holes.  I assume you mean that just
because head.S maps a chunk of memory, it isn't necessarily available
for allocation because its a hole.  But does alloc_bootmem know to avoid
them anyway?  Has it already parsed e820 at that point?

J
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread H. Peter Anvin

Jeremy Fitzhardinge wrote:

Eric W. Biederman wrote:

The only way to ensure this will not happen is to do what we do
on x86_64 and map the new page table page into our address space
before we write to it.  Assuming the page we allocate is already
mapped is simply not robust.
  


So you mean make alloc_bootmem make sure there's a mapping for the
returned page?  That would be simple enough, though it might interact
strangely with the actual construction of the memory mappings (what if
the page alloc_bootmem just allocated for the pagetable is the page
pagetable_init is actually trying to map?).

I'm not sure about the issue with holes.  I assume you mean that just
because head.S maps a chunk of memory, it isn't necessarily available
for allocation because its a hole.  But does alloc_bootmem know to avoid
them anyway?  Has it already parsed e820 at that point?



Since we allocate the maximum possible memory statically, I fail to see 
how holes could make the situation any worse, or better.


-hpa
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Jeremy Fitzhardinge
H. Peter Anvin wrote:
 Since we allocate the maximum possible memory statically, I fail to
 see how holes could make the situation any worse, or better. 

No, we map enough space to map 4G (~4 pages), but we don't actually map
4G.  If a hole happened to start within that 4 page mapping, then the
memory still wouldn't be available for allocation.

I think this is a bit of a spurious argument though, since if it were
really a problem we'd have to worry about holes hitting the kernel image
too.  As far as I can see, that's not considered to be a problem.

I think the real point is that there's currently a subtle dependency
between head.S and bootmem allocation which happens between start_kernel
and pagetable_init.  Your patch preventing over-mapping should make them
easier to smoke out as it currently stands, but eliminating the problem
by making alloc_bootmem create the mappings for itself does have
appeal.  There would still be the dependency on head.S to map the kernel
itself and the bootmem allocator bitmap.

J
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Eric W. Biederman
H. Peter Anvin [EMAIL PROTECTED] writes:

 Since we allocate the maximum possible memory statically, I fail to see how
 holes could make the situation any worse, or better.

Consider a memory hole of size 8M immediately after our bootmem bitmap.
head.S which knows nothing of holes will map the pages of the hole
into the initial page tables assuming that is where the page tables
will live.

alloc_bootmem_low_pages which understands holes will skip those pages
and return a higher address, which is outside of our initial mapping
of pages.

As for how to fix this, we need to call boot_ioremap or better
bt_ioremap on the address returned from alloc_bootmem_pages_low to
force the allocated page into the page table.  After we have setup
the page table page we can call unmap the page to free up the scarce
early mapping resource.

Eric
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread H. Peter Anvin

Jeremy Fitzhardinge wrote:

H. Peter Anvin wrote:

Since we allocate the maximum possible memory statically, I fail to
see how holes could make the situation any worse, or better. 


No, we map enough space to map 4G (~4 pages), but we don't actually map
4G.  If a hole happened to start within that 4 page mapping, then the
memory still wouldn't be available for allocation.

I think this is a bit of a spurious argument though, since if it were
really a problem we'd have to worry about holes hitting the kernel image
too.  As far as I can see, that's not considered to be a problem.

I think the real point is that there's currently a subtle dependency
between head.S and bootmem allocation which happens between start_kernel
and pagetable_init.  Your patch preventing over-mapping should make them
easier to smoke out as it currently stands, but eliminating the problem
by making alloc_bootmem create the mappings for itself does have
appeal.  There would still be the dependency on head.S to map the kernel
itself and the bootmem allocator bitmap.



Agreed.  However, saying that your patch shouldn't go into the mainline 
kernel until that has been fixed is spurious and wrong.  It fixes a real 
problem with minimal risk.


-hpa
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Eric W. Biederman
Jeremy Fitzhardinge [EMAIL PROTECTED] writes:

 H. Peter Anvin wrote:
 Since we allocate the maximum possible memory statically, I fail to
 see how holes could make the situation any worse, or better. 

 No, we map enough space to map 4G (~4 pages), but we don't actually map
 4G.  If a hole happened to start within that 4 page mapping, then the
 memory still wouldn't be available for allocation.

 I think this is a bit of a spurious argument though, since if it were
 really a problem we'd have to worry about holes hitting the kernel image
 too.  As far as I can see, that's not considered to be a problem.

- holes hitting the kernel image is not something we can do anything about.

- I have always believed we need to export enough information so the
  bootloader can verify that we don't hit a hole in the initial
  kernel image.

- I know of one system that had BIOS tables at 16MB I believe (and
  thus had a fairly low hole).

- Given that we are actually increasing (not decreasing) the number of
  scenarios where we boot the kernel it probably makes sense to be
  as robust as we can.

- If we support putting the ramdisk immediately after the kernel image
  in memory we will actually hit this case in practice. 

 I think the real point is that there's currently a subtle dependency
 between head.S and bootmem allocation which happens between start_kernel
 and pagetable_init.  Your patch preventing over-mapping should make them
 easier to smoke out as it currently stands, but eliminating the problem
 by making alloc_bootmem create the mappings for itself does have
 appeal.  There would still be the dependency on head.S to map the kernel
 itself and the bootmem allocator bitmap.

Agreed.  However that is essentially all statically allocated memory
and the best we can do at this point in time.

Eric
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread H. Peter Anvin

Eric W. Biederman wrote:


- I know of one system that had BIOS tables at 16MB I believe (and
  thus had a fairly low hole).



Please name names, otherwise this is just rumouring.  Seriously.  We 
have enough cargo-cult programming as it is.


A lot of old ISA systems had an option to put a memory hole at 15-16 MB 
to support ISA video cards with linear frame buffer.  It very rarely got 
used, since VLB took over around the same time.


-hpa
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Eric W. Biederman
H. Peter Anvin [EMAIL PROTECTED] writes:

 Eric W. Biederman wrote:

 - I know of one system that had BIOS tables at 16MB I believe (and
   thus had a fairly low hole).


 Please name names, otherwise this is just rumouring.  Seriously.  We have 
 enough
 cargo-cult programming as it is.

The unisys systems.  I don't remember exactly where they put there ACPI
tables but it was low enough that the default address for the kdump kernels
conflicted with it.  So this is likely 16MB or 32MB

 A lot of old ISA systems had an option to put a memory hole at 15-16 MB to
 support ISA video cards with linear frame buffer.  It very rarely got used,
 since VLB took over around the same time.

No this was a much larger and more modern system.

Eric
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Andi Kleen
On Monday 23 April 2007 19:45:41 H. Peter Anvin wrote:
 Eric W. Biederman wrote:
  
  - I know of one system that had BIOS tables at 16MB I believe (and
thus had a fairly low hole).
  
 
 Please name names, otherwise this is just rumouring.  Seriously.  We 
 have enough cargo-cult programming as it is.

Unisys did this at some point in their large machines, but they fixed
that in a later BIOS.

Anyways, boot up should be robust against any holes if possible.

-Andi

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Eric W. Biederman
H. Peter Anvin [EMAIL PROTECTED] writes:

 Agreed.  However, saying that your patch shouldn't go into the mainline kernel
 until that has been fixed is spurious and wrong.  It fixes a real problem with
 minimal risk.

For a stable and frozen kernel it is probably the best we can do.

However the code is based on confusion of what should be happening and
it increases the complexity of the code and pushes us towards the edge of
code that is so crufty it is impossible to maintain.

We need to simplify this mess not make it more convoluted.

Did anyone else notice that if we don't have PAE we continue to use
the initial page tables 4K page mappings of the kernel?

This probably explains why experiments to 4MB align the i386 kernel
to improve performance failed.

Eric
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Jeremy Fitzhardinge
Eric W. Biederman wrote:
 Consider a memory hole of size 8M immediately after our bootmem bitmap.
 head.S which knows nothing of holes will map the pages of the hole
 into the initial page tables assuming that is where the page tables
 will live.
   

Sure, but considering we're only talking about mapping an additional *2*
pages, the chances are that it will affect a kernel as much as the early
head.S mapping.  It's well within the variability of fairly similar
kernel configs.

 As for how to fix this, we need to call boot_ioremap or better
 bt_ioremap on the address returned from alloc_bootmem_pages_low to
 force the allocated page into the page table.  After we have setup
 the page table page we can call unmap the page to free up the scarce
 early mapping resource.
   

Would that be necessary?  Is there any need to remap it?  Couldn't you
just do the straightforward phys-virt mapping and use the page where it
lies?

J
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Eric W. Biederman
Jeremy Fitzhardinge [EMAIL PROTECTED] writes:

 Eric W. Biederman wrote:
 Consider a memory hole of size 8M immediately after our bootmem bitmap.
 head.S which knows nothing of holes will map the pages of the hole
 into the initial page tables assuming that is where the page tables
 will live.
   

 Sure, but considering we're only talking about mapping an additional *2*
 pages, the chances are that it will affect a kernel as much as the early
 head.S mapping.  It's well within the variability of fairly similar
 kernel configs.

If (cpu_has_pse) it may only be an additional two pages.
INIT_MAP_BEYOND is currently mapping a lot more then that.

 As for how to fix this, we need to call boot_ioremap or better
 bt_ioremap on the address returned from alloc_bootmem_pages_low to
 force the allocated page into the page table.  After we have setup
 the page table page we can call unmap the page to free up the scarce
 early mapping resource.
   

 Would that be necessary?  Is there any need to remap it?  Couldn't you
 just do the straightforward phys-virt mapping and use the page where it
 lies?

Catch-22.  If the page is not mapped yet you can't use it where it lies.

Eric
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Jeremy Fitzhardinge
Eric W. Biederman wrote:
 If (cpu_has_pse) it may only be an additional two pages.
 INIT_MAP_BEYOND is currently mapping a lot more then that.
   

Ah, yes.  It allocates an extra two pages for pagetables, and then maps
an extra 8MB or so.

 Would that be necessary?  Is there any need to remap it?  Couldn't you
 just do the straightforward phys-virt mapping and use the page where it
 lies?
 

 Catch-22.  If the page is not mapped yet you can't use it where it lies.

Why not?  Er, except in the case where the page is needed to map itself
- but that can be dealt with with a transient fixmap mapping.

J
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread H. Peter Anvin

Jeremy Fitzhardinge wrote:


Why not?  Er, except in the case where the page is needed to map itself
- but that can be dealt with with a transient fixmap mapping.



It would be *trivial* to make a certain number of page table slots 
available at the end of the head.S-generated map.


-hpa
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Jeremy Fitzhardinge
H. Peter Anvin wrote:
 It would be *trivial* to make a certain number of page table slots
 available at the end of the head.S-generated map. 

Or you could use a fixmap.

J
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread Eric W. Biederman
Jeremy Fitzhardinge [EMAIL PROTECTED] writes:

 H. Peter Anvin wrote:
 It would be *trivial* to make a certain number of page table slots
 available at the end of the head.S-generated map. 

 Or you could use a fixmap.

That certain number of page table slots should be the fixmap slots.
If you do that bt_ioremap works, we remove a boatload of special cases
and the code becomes simpler.

Eric
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-23 Thread H. Peter Anvin

Eric W. Biederman wrote:

Jeremy Fitzhardinge [EMAIL PROTECTED] writes:


H. Peter Anvin wrote:

It would be *trivial* to make a certain number of page table slots
available at the end of the head.S-generated map. 

Or you could use a fixmap.


That certain number of page table slots should be the fixmap slots.
If you do that bt_ioremap works, we remove a boatload of special cases
and the code becomes simpler.


It's probably worth noting that several of the non-PC ports already do 
exactly this, although in a hacky way.  Grep for pg0 and you'll find it.


-hpa
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-19 Thread Jeremy Fitzhardinge
Chuck Ebbert wrote:
> H. Peter Anvin wrote:
>   
>> Andi Kleen wrote:
>> 
>>> Then we would have seen reports surely?
>>>   

Yes, I would have thought so.  It surprised me that such an obvious bug
could be there, apparently for a long time.  But it's real, and
potentially affects everyone.  It probably doesn't affect highly modular
distros much, since the kernel itself will be relatively small.

> I never saw a description of the symptoms of encountering this bug.
> Does it just hang, or what?
>   

You get an early-fault message on-screen, assuming that's enabled;
otherwise it will just appear to hang.  It happens in pagetable_init,
when it allocates a new pagetable above the head.S mapping (8M in my
case).  It will only hit if the kernel size approaches a 4M boundary,
since it won't leave enough space mapped to construct the lowmem mappings.

It only affects native booting, since under Xen all those mappings have
already been constructed.  It happened to me with a paravirt kernel that
happened to Xen compiled into it, but that was irrelevent (though
misleading; the 40k difference in kernel size was enough to make it not
happen in a non-Xen kernel).

J
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-19 Thread Chuck Ebbert
H. Peter Anvin wrote:
> Andi Kleen wrote:
>> On Thursday 19 April 2007 22:55:50 H. Peter Anvin wrote:
>>> Andi Kleen wrote:
> Is some version of this going in for 2.6.21, or is it not a real
> problem?
 When it's only seen with Xen it's not a real problem right now.
>>> It's not just seen only with Xen, though.  It will affect all kernels
>>> in a particular range of sizes, and we have ordinary kernels covering
>>> that range (in fact, Xen hardly affects size in that way.)
>>
>> Then we would have seen reports surely?
> 
> The critical size window is relatively small, though.  The size of the
> uncompressed kernel mod 4 MB must be above 3.75 MB or so.  If you have
> less than 1 GB RAM, the window is even smaller.  This is strictly
> configuration-dependent, and has nothing to do with Xen.  It just so
> happened that the first people to detect and diagnose it (as opposed to
> just "it doesn't boot") were Xen people.

I never saw a description of the symptoms of encountering this bug.
Does it just hang, or what?

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-19 Thread H. Peter Anvin

Andi Kleen wrote:

On Thursday 19 April 2007 22:55:50 H. Peter Anvin wrote:

Andi Kleen wrote:

Is some version of this going in for 2.6.21, or is it not a real problem?

When it's only seen with Xen it's not a real problem right now.
It's not just seen only with Xen, though.  It will affect all kernels in 
a particular range of sizes, and we have ordinary kernels covering that 
range (in fact, Xen hardly affects size in that way.)


Then we would have seen reports surely?


The critical size window is relatively small, though.  The size of the 
uncompressed kernel mod 4 MB must be above 3.75 MB or so.  If you have 
less than 1 GB RAM, the window is even smaller.  This is strictly 
configuration-dependent, and has nothing to do with Xen.  It just so 
happened that the first people to detect and diagnose it (as opposed to 
just "it doesn't boot") were Xen people.


-hpa
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-19 Thread Andi Kleen
On Thursday 19 April 2007 22:55:50 H. Peter Anvin wrote:
> Andi Kleen wrote:
> >> Is some version of this going in for 2.6.21, or is it not a real problem?
> > 
> > When it's only seen with Xen it's not a real problem right now.
> 
> It's not just seen only with Xen, though.  It will affect all kernels in 
> a particular range of sizes, and we have ordinary kernels covering that 
> range (in fact, Xen hardly affects size in that way.)

Then we would have seen reports surely?

-Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-19 Thread H. Peter Anvin

Andi Kleen wrote:

Is some version of this going in for 2.6.21, or is it not a real problem?


When it's only seen with Xen it's not a real problem right now.


It's not just seen only with Xen, though.  It will affect all kernels in 
a particular range of sizes, and we have ordinary kernels covering that 
range (in fact, Xen hardly affects size in that way.)


-hpa
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-19 Thread Andi Kleen

> 
> Is some version of this going in for 2.6.21, or is it not a real problem?

When it's only seen with Xen it's not a real problem right now.

-Andi


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-19 Thread Chuck Ebbert
Jeremy Fitzhardinge wrote:
> head.S creates the very initial pagetable for the kernel.  This just
> maps enough space for the kernel itself, and an allocation bitmap.
> The amount of mapped memory is rounded up to 4Mbytes, and so this
> typically ends up mapping 8Mbytes of memory.
> 
> When booting, pagetable_init() needs to create mappings for all
> lowmem, and the pagetables for these mappings are allocated from the
> free pages around the kernel in low memory.  If the number of
> pagetable pages + kernel size exceeds head.S's initial mapping, it
> will end up faulting on an unmapped page.  This will only happen with
> specific combinations of kernel size and memory size.
> 
> This patch makes sure that head.S also maps enough space to fit the
> kernel pagetables as well as the kernel itself.  It ends up using an
> additional two pages of unreclaimable memory.
> 
> Signed-off-by: Jeremy Fitzhardinge <[EMAIL PROTECTED]>
> Acked-by: "H. Peter Anvin" <[EMAIL PROTECTED]>
> Cc: Andi Kleen <[EMAIL PROTECTED]>
> Cc: Zachary Amsden <[EMAIL PROTECTED]>
> Cc: Chris Wright <[EMAIL PROTECTED]>
> Cc: "Eric W. Biederman" <[EMAIL PROTECTED]>
> Cc: Linus Torvalds <[EMAIL PROTECTED]>,

Is some version of this going in for 2.6.21, or is it not a real problem?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-19 Thread Chuck Ebbert
Jeremy Fitzhardinge wrote:
 head.S creates the very initial pagetable for the kernel.  This just
 maps enough space for the kernel itself, and an allocation bitmap.
 The amount of mapped memory is rounded up to 4Mbytes, and so this
 typically ends up mapping 8Mbytes of memory.
 
 When booting, pagetable_init() needs to create mappings for all
 lowmem, and the pagetables for these mappings are allocated from the
 free pages around the kernel in low memory.  If the number of
 pagetable pages + kernel size exceeds head.S's initial mapping, it
 will end up faulting on an unmapped page.  This will only happen with
 specific combinations of kernel size and memory size.
 
 This patch makes sure that head.S also maps enough space to fit the
 kernel pagetables as well as the kernel itself.  It ends up using an
 additional two pages of unreclaimable memory.
 
 Signed-off-by: Jeremy Fitzhardinge [EMAIL PROTECTED]
 Acked-by: H. Peter Anvin [EMAIL PROTECTED]
 Cc: Andi Kleen [EMAIL PROTECTED]
 Cc: Zachary Amsden [EMAIL PROTECTED]
 Cc: Chris Wright [EMAIL PROTECTED]
 Cc: Eric W. Biederman [EMAIL PROTECTED]
 Cc: Linus Torvalds [EMAIL PROTECTED],

Is some version of this going in for 2.6.21, or is it not a real problem?
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-19 Thread Andi Kleen

 
 Is some version of this going in for 2.6.21, or is it not a real problem?

When it's only seen with Xen it's not a real problem right now.

-Andi


-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-19 Thread H. Peter Anvin

Andi Kleen wrote:

Is some version of this going in for 2.6.21, or is it not a real problem?


When it's only seen with Xen it's not a real problem right now.


It's not just seen only with Xen, though.  It will affect all kernels in 
a particular range of sizes, and we have ordinary kernels covering that 
range (in fact, Xen hardly affects size in that way.)


-hpa
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-19 Thread Andi Kleen
On Thursday 19 April 2007 22:55:50 H. Peter Anvin wrote:
 Andi Kleen wrote:
  Is some version of this going in for 2.6.21, or is it not a real problem?
  
  When it's only seen with Xen it's not a real problem right now.
 
 It's not just seen only with Xen, though.  It will affect all kernels in 
 a particular range of sizes, and we have ordinary kernels covering that 
 range (in fact, Xen hardly affects size in that way.)

Then we would have seen reports surely?

-Andi
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-19 Thread H. Peter Anvin

Andi Kleen wrote:

On Thursday 19 April 2007 22:55:50 H. Peter Anvin wrote:

Andi Kleen wrote:

Is some version of this going in for 2.6.21, or is it not a real problem?

When it's only seen with Xen it's not a real problem right now.
It's not just seen only with Xen, though.  It will affect all kernels in 
a particular range of sizes, and we have ordinary kernels covering that 
range (in fact, Xen hardly affects size in that way.)


Then we would have seen reports surely?


The critical size window is relatively small, though.  The size of the 
uncompressed kernel mod 4 MB must be above 3.75 MB or so.  If you have 
less than 1 GB RAM, the window is even smaller.  This is strictly 
configuration-dependent, and has nothing to do with Xen.  It just so 
happened that the first people to detect and diagnose it (as opposed to 
just it doesn't boot) were Xen people.


-hpa
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-19 Thread Chuck Ebbert
H. Peter Anvin wrote:
 Andi Kleen wrote:
 On Thursday 19 April 2007 22:55:50 H. Peter Anvin wrote:
 Andi Kleen wrote:
 Is some version of this going in for 2.6.21, or is it not a real
 problem?
 When it's only seen with Xen it's not a real problem right now.
 It's not just seen only with Xen, though.  It will affect all kernels
 in a particular range of sizes, and we have ordinary kernels covering
 that range (in fact, Xen hardly affects size in that way.)

 Then we would have seen reports surely?
 
 The critical size window is relatively small, though.  The size of the
 uncompressed kernel mod 4 MB must be above 3.75 MB or so.  If you have
 less than 1 GB RAM, the window is even smaller.  This is strictly
 configuration-dependent, and has nothing to do with Xen.  It just so
 happened that the first people to detect and diagnose it (as opposed to
 just it doesn't boot) were Xen people.

I never saw a description of the symptoms of encountering this bug.
Does it just hang, or what?

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-19 Thread Jeremy Fitzhardinge
Chuck Ebbert wrote:
 H. Peter Anvin wrote:
   
 Andi Kleen wrote:
 
 Then we would have seen reports surely?
   

Yes, I would have thought so.  It surprised me that such an obvious bug
could be there, apparently for a long time.  But it's real, and
potentially affects everyone.  It probably doesn't affect highly modular
distros much, since the kernel itself will be relatively small.

 I never saw a description of the symptoms of encountering this bug.
 Does it just hang, or what?
   

You get an early-fault message on-screen, assuming that's enabled;
otherwise it will just appear to hang.  It happens in pagetable_init,
when it allocates a new pagetable above the head.S mapping (8M in my
case).  It will only hit if the kernel size approaches a 4M boundary,
since it won't leave enough space mapped to construct the lowmem mappings.

It only affects native booting, since under Xen all those mappings have
already been constructed.  It happened to me with a paravirt kernel that
happened to Xen compiled into it, but that was irrelevent (though
misleading; the 40k difference in kernel size was enough to make it not
happen in a non-Xen kernel).

J
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-15 Thread Andreas Schwab
Jan Engelhardt <[EMAIL PROTECTED]> writes:

> LOW_PAGES = (0x1ULL - __PAGE_OFFSET) >> PAGE_SHIT_asm

The assembler does not now anything about ULL.

Andreas.

-- 
Andreas Schwab, SuSE Labs, [EMAIL PROTECTED]
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-15 Thread Jan Engelhardt

On Apr 14 2007 15:04, H. Peter Anvin wrote:
>
> Jeremy Fitzhardinge wrote:
>> -
>> +LOW_PAGES = 1<<(32-PAGE_SHIFT_asm)
>> +
>
> Again, for debugging... it would be interesting to replace this with:
>
> LOW_PAGES = (0x1-__PAGE_OFFSET) >> PAGE_SHIFT_asm

LOW_PAGES = (0x1ULL - __PAGE_OFFSET) >> PAGE_SHIT_asm


Jan
-- 
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-15 Thread Jan Engelhardt

On Apr 14 2007 15:04, H. Peter Anvin wrote:

 Jeremy Fitzhardinge wrote:
 -
 +LOW_PAGES = 1(32-PAGE_SHIFT_asm)
 +

 Again, for debugging... it would be interesting to replace this with:

 LOW_PAGES = (0x1-__PAGE_OFFSET)  PAGE_SHIFT_asm

LOW_PAGES = (0x1ULL - __PAGE_OFFSET)  PAGE_SHIT_asm


Jan
-- 
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-15 Thread Andreas Schwab
Jan Engelhardt [EMAIL PROTECTED] writes:

 LOW_PAGES = (0x1ULL - __PAGE_OFFSET)  PAGE_SHIT_asm

The assembler does not now anything about ULL.

Andreas.

-- 
Andreas Schwab, SuSE Labs, [EMAIL PROTECTED]
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
And now for something completely different.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-14 Thread H. Peter Anvin

Jeremy Fitzhardinge wrote:

-
+LOW_PAGES = 1<<(32-PAGE_SHIFT_asm)
+


Again, for debugging... it would be interesting to replace this with:

LOW_PAGES = (0x1-__PAGE_OFFSET) >> PAGE_SHIFT_asm

... to smoke out further problems; this will take the strict definition 
of "lowmem" (modulo the pci region, which someone unwisely made 
dynamically adjustable... which would have been fine if initramfs could 
live in highmem, which it can't yet.)


-hpa
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 10/28] i386: map enough initial memory to create lowmem mappings

2007-04-14 Thread H. Peter Anvin

Jeremy Fitzhardinge wrote:

-
+LOW_PAGES = 1(32-PAGE_SHIFT_asm)
+


Again, for debugging... it would be interesting to replace this with:

LOW_PAGES = (0x1-__PAGE_OFFSET)  PAGE_SHIFT_asm

... to smoke out further problems; this will take the strict definition 
of lowmem (modulo the pci region, which someone unwisely made 
dynamically adjustable... which would have been fine if initramfs could 
live in highmem, which it can't yet.)


-hpa
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/