Re: [coreboot] [LinuxBIOS] Messing with CAR in a big way
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi all,
> K8 is a bit different than Intel and the Geode in how CAR works. For the
> K8 the cache is always left enabled. I think that means that the tags
> should all be correct and that a copy shouldn't be required, just the
> wbinvd. There has been some discussion about wbind and invd. I have
> always thought that a wbind should work as long as there is memory
> backing that area in cache. I think someone just needs to try it.
I checked all documentation and it seems writeback does not work on pre rev F.
of CPUs. Cache must be always invalidated.
Hmm I have some troubles on Opteron 175. I switched Coreboot to _RAMBASE=3MB and
TOPK to 4096, I also fixed the initial cleaning routine to clean just
clear_memory(_RAMBASE, (CONFIG_LB_MEM_TOPK<<10) - DCACHE_RAM_SIZE) );
But CPU never returns from clear_init_ram, maybe some tags get b0rked.
I tried the copy-on-same-place trick and removed the stack relocation and it
worked too.
My code looks like this now:
disable_cache_as_ram_bsp();
memcopy((void *)DCACHE_RAM_BASE , (void *)DCACHE_RAM_BASE, DCACHE_RAM_SIZE);
__asm__("invd\n");
clear_init_ram();
Also I applied the cleanup patch for cache_as_ram.inc.
Seems to work, except there must be some other problem with resource management.
When:
Reading resources
all entries are listed, but no "Done". I think I have seen this problem already
when I removed the call for clear_memory some time ago so something must use
some untitialized memory from 0-3MB which is strange. I think it will get work
again if I just add clear 0-640KB. I will try before sleep now.
Rudolf
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFHi+la3J9wPJqZRNURAvgFAKDUg64W+U+NvhhGRq0hVlXZeBxoWgCcDABU
C8MHQjKDILAtxeUUgZgkQOs=
=FSfb
-END PGP SIGNATURE-
--
coreboot mailing list
coreboot@coreboot.org
http://www.coreboot.org/mailman/listinfo/coreboot
Re: [coreboot] [LinuxBIOS] Messing with CAR in a big way
ron minnich wrote:
> For reference here is the geode lx disable car. CAR is at 8. The
> steps are to re-enable cache (for loop), then dirty all the tags for
> the CAR area (cld; rep movsl; etc); then write it all back (wbinvd).
>
> Works really well.
>
> for (i = 0; i < ARRAY_SIZE(msr_table); i++)
> wrmsr(msr_table[i].msrnum, msr_table[i].msr);
>
> __asm__ __volatile__("cld; rep movsl" ::"D" (DCACHE_RAM_BASE),
> "S" (DCACHE_RAM_BASE), "c" (DCACHE_RAM_SIZE/4): "memory");
>
> __asm__ __volatile__ ("wbinvd\n");
>
> Would be great if this worked on k8.
>
> Thanks ron
>
>
K8 is a bit different than Intel and the Geode in how CAR works. For the
K8 the cache is always left enabled. I think that means that the tags
should all be correct and that a copy shouldn't be required, just the
wbinvd. There has been some discussion about wbind and invd. I have
always thought that a wbind should work as long as there is memory
backing that area in cache. I think someone just needs to try it.
Marc
--
Marc Jones
Senior Firmware Engineer
(970) 226-9684 Office
mailto:[EMAIL PROTECTED]
http://www.amd.com/embeddedprocessors
--
coreboot mailing list
coreboot@coreboot.org
http://www.coreboot.org/mailman/listinfo/coreboot
Re: [coreboot] [LinuxBIOS] Messing with CAR in a big way / S3 considerations
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 > Interesting. I must have overlooked that part. Nope, get new revision http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/32559.pdf 32559 Rev. 3.08 July 2007 Rudolf -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHiW+r3J9wPJqZRNURAimWAJ9SVMiKhvG7FB/us2mKWNfk+RZO+ACeKFNh 3ReSodBaBgQvDrK+qxVxlD0= =sNiS -END PGP SIGNATURE- -- coreboot mailing list coreboot@coreboot.org http://www.coreboot.org/mailman/listinfo/coreboot
Re: [coreboot] [LinuxBIOS] Messing with CAR in a big way / S3 considerations
On 12.01.2008 23:57, Rudolf Marek wrote: > > Well, CAR has to be below 1 MB (processor limitation). We can make sure > > it will not clobber more than 64 kB in that area. Because of conflicts > > with legacy ROM decoded areas, CAR has to stay below 896 kB. Conflicts > > with areas decoded by devices are pretty likely in the 640 kB - 1024 kB > > range. > > Hum OK, so lets use 0-64KB? That could work for processors which are guaranteed to have more than 64 kB of L2 cache. Basically, you can place the CAR area with 64 kB size granularity and alignment between 0 kB and 512 kB, with 16 kB granularity+alignment between 512 kB and 768 kB, and with 4 kB granularity+alignment between 768 kB and 1024 kB. That is a processor limitation. Now if a processor has 64 kB L2 cache or less, placing CAR between 0 kB and 512 kB is naturally impossible. /"You are in a maze of twisty little passages, all alike." /Of course, CAR is rather fragile and there is no guarantee my rewrites will ever allow us to use the full theoretical potential. > > Well, during CAR our stack has to be below 1 MB. Moving the stack during > > execution is something we do now for K8, but I'd like to get rid of that > > completely so we never introduce such stuff to v3. Note: Geode LX in v3 > > does not move the stack. > > Ok when it is possible to remap the stack to same location (as opposed > to what > the Opteron BKDG says) > > Up to 0xe AMD family it says that cache must be invalidated with INVD and > writeback is not possible with CLFLUSH or WBINVB. Maybe Manual > writeback is then. > > 0xf AMD family allows only 48KB, 256 tags must be reserved. > It says nothing about "unable to copy it on itself". Only that cache > invalidate must be done after DRAM works. Yes. > And new guide revision also says: Temporarily disable cache fill > probes on the > BSP by programming DisFillP = 1 (Function 0, offset 68h). Then read > 64K and > enable the DisFillP = 0. It does not tell if data can be written back with > WINVD, CLFLUSH just to avoid that before dram is ready. Interesting. I must have overlooked that part. > the 0x10 fam BKDG says even more, also just 48KB, read all with rep MOV, > there are additional constrains like not to push more then a double word > with first push. Yes, fragility abounds. > It also says that writeback to main memory is possible with CLFLUSH > or WBINVD > invd. It also recommends not to use exceptions, interrupts, no MMX, > 3Dow etc > Except of: MOVD, MOVQ, MOVDQA, MOVQ2DQ, MOVDQ2Q. > > Maybe manual writeback will work here too. Let's try it. Regards, Carl-Daniel -- coreboot mailing list coreboot@coreboot.org http://www.coreboot.org/mailman/listinfo/coreboot
Re: [coreboot] [LinuxBIOS] Messing with CAR in a big way / S3 considerations
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all, > Well, CAR has to be below 1 MB (processor limitation). We can make sure > it will not clobber more than 64 kB in that area. Because of conflicts > with legacy ROM decoded areas, CAR has to stay below 896 kB. Conflicts > with areas decoded by devices are pretty likely in the 640 kB - 1024 kB > range. Hum OK, so lets use 0-64KB? > >>> What is the reason that we can't put all coreboot >>> variables below 1MB? >> Yes we can, but some parts are very messy including but not limited to: >> >> 1) stack segment for AP cpus, very ugly define, hardwired to 1MB when >> using >> _RAMBASE lower then 1MB >> >> 2) Memory is cleared from zero to TOPK, which is bad. > > But the two points above are pure code issues and can be fixed, right? Yes it could of course. IMHO there is no much space left for everything bellow 1MB so I think there is a hack with the 1MB above usage for the stack. > > An area with 64 kB size somewhere below 1 MB should cover variable > storage. We might have to mess with the 0-4 kB area as well. I think this is fine, why not, because the LB table is there written once Coreboot is nearly done anyway. Lets put there secondary.S trampoline and back to real mode switcher too. > Well, during CAR our stack has to be below 1 MB. Moving the stack during > execution is something we do now for K8, but I'd like to get rid of that > completely so we never introduce such stuff to v3. Note: Geode LX in v3 > does not move the stack. Ok when it is possible to remap the stack to same location (as opposed to what the Opteron BKDG says) Up to 0xe AMD family it says that cache must be invalidated with INVD and writeback is not possible with CLFLUSH or WBINVB. Maybe Manual writeback is then. 0xf AMD family allows only 48KB, 256 tags must be reserved. It says nothing about "unable to copy it on itself". Only that cache invalidate must be done after DRAM works. And new guide revision also says: Temporarily disable cache fill probes on the BSP by programming DisFillP = 1 (Function 0, offset 68h). Then read 64K and enable the DisFillP = 0. It does not tell if data can be written back with WINVD, CLFLUSH just to avoid that before dram is ready. the 0x10 fam BKDG says even more, also just 48KB, read all with rep MOV, there are additional constrains like not to push more then a double word with first push. It also says that writeback to main memory is possible with CLFLUSH or WBINVD invd. It also recommends not to use exceptions, interrupts, no MMX, 3Dow etc Except of: MOVD, MOVQ, MOVDQA, MOVQ2DQ, MOVDQ2Q. Maybe manual writeback will work here too. > Congratulations! Thanks, yep, its cool ;) Rudolf -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHiUXY3J9wPJqZRNURAok9AJ93Gi7OywIVfAv1A2f+WI5C6BukCwCfc9Le 9ktjrwY4ZUYkoJZxnUepbvE= =/CVa -END PGP SIGNATURE- -- coreboot mailing list coreboot@coreboot.org http://www.coreboot.org/mailman/listinfo/coreboot
Re: [coreboot] [LinuxBIOS] Messing with CAR in a big way / S3 considerations
On 12.01.2008 22:39, Rudolf Marek wrote: > ron minnich wrote: > > Rudolf, many payloads start at 1 MB. I don't think we want to use CAR > > at that low address. > > Maybe if you want to have CAR address = RAM address (to copy it to itself) > Just please choose TOPK - CAR_SIZE as it is now. Well, CAR has to be below 1 MB (processor limitation). We can make sure it will not clobber more than 64 kB in that area. Because of conflicts with legacy ROM decoded areas, CAR has to stay below 896 kB. Conflicts with areas decoded by devices are pretty likely in the 640 kB - 1024 kB range. > > What is the reason that we can't put all coreboot > > variables below 1MB? > > Yes we can, but some parts are very messy including but not limited to: > > 1) stack segment for AP cpus, very ugly define, hardwired to 1MB when > using > _RAMBASE lower then 1MB > > 2) Memory is cleared from zero to TOPK, which is bad. But the two points above are pure code issues and can be fixed, right? > > Just trying to understand s3 :-) > > Before suspend phase: > Coreboot boots payload, it boots OS. > > Suspend phase: > Linux will put to a ACPI table a waking vector address, and uses the > value from > DSDT to write to SB value to activate the S3 sleep. > > Resume phase: Coreboot runs as from coldboot except of taking memory from > self-refresh mode. No payload is to be loaded, only it is jumped back > to OS. > > What it needs: > > It just needs not to mess with memory too much. We can reserve a > region for > Coreboot say at 1MB-2MB (plus usual reserved regions like 0-4K and > 960K-1MB) See my list of constraints above. > I dont want to take all low memory because the entry point is in real > mode and > the OS must have some memory bellow 1MB free. An area with 64 kB size somewhere below 1 MB should cover variable storage. We might have to mess with the 0-4 kB area as well. > There are parts of Coreboot which are messing too much with memory, like I > described in previous mails and this must be fixed. Easiest way how to > avoid > non-std solutions and get instantly more memory is to escape to up 1MB > of memory > or to any better place ;) I dont think it is wise to have for LB only > 0 - 640KB > of RAM with lot of exceptions, lets use one continuous block in > "higher" RAM. > ACPI says, reserved blocks can start at 8MB. > > Problematic parts: > > Secondary.S - (the trampoline for AP CPUs) > > lapic_cpu_init.c - (it copies the Secondary.S to undefined place in > low memory, > and AP cpu stack is handled badly when _RAMBASE is bellow 1MB) > > wakeup.c (yeah mine implentation which jumps back to real mode, this > must be > fixed, so no data at our reserved region of 0-4KB gets overwritten) > > clear_init_ram.c (in AMD CAR) > it just clears a lot of memory from ZERO instead of _RAMBASE to TOPK > > Used CAR "memory/cache" should not be a problem, because it is not > written back > to memory in any way. > > So to sum it up. If we would isolate Coreboot writes to memory to > those regions: > > 0-4K > > 960K-1M > > _RAMBASE - TOPK (RAMBASE > 1MB) > > It would be nice, and just _RAMBASE - TOPK region could be reserved in > memory > map, because it uses Coreboot when starting after resume. This memory > region > could be claimed back to OS via ACPI memory hotplug, which I'm now > investigating. Reserving permanently 1MB of memory when using S3 would > be also > possible because Asus BIOS takes 1MB at the end of memory for SMM and > suspend too. Well, during CAR our stack has to be below 1 MB. Moving the stack during execution is something we do now for K8, but I'd like to get rid of that completely so we never introduce such stuff to v3. Note: Geode LX in v3 does not move the stack. > I hope it is now clear how it works ;) I just avoided in my > experiments all > those stuff because I told linux to use 2MB-1024MB :) (my TOPK is 2048) Nice trick. > But now I really would like to get this solved too, because suspend to > RAM now > works which is great leap because with proprietary BIOS I was unable > to get it > working :))) Congratulations! Regards, Carl-Daniel -- coreboot mailing list coreboot@coreboot.org http://www.coreboot.org/mailman/listinfo/coreboot
Re: [coreboot] [LinuxBIOS] Messing with CAR in a big way / S3 considerations
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 ron minnich wrote: > Rudolf, many payloads start at 1 MB. I don't think we want to use CAR > at that low address. Maybe if you want to have CAR address = RAM address (to copy it to itself) Just please choose TOPK - CAR_SIZE as it is now. > What is the reason that we can't put all coreboot > variables below 1MB? Yes we can, but some parts are very messy including but not limited to: 1) stack segment for AP cpus, very ugly define, hardwired to 1MB when using _RAMBASE lower then 1MB 2) Memory is cleared from zero to TOPK, which is bad. > Just trying to understand s3 :-) Before suspend phase: Coreboot boots payload, it boots OS. Suspend phase: Linux will put to a ACPI table a waking vector address, and uses the value from DSDT to write to SB value to activate the S3 sleep. Resume phase: Coreboot runs as from coldboot except of taking memory from self-refresh mode. No payload is to be loaded, only it is jumped back to OS. What it needs: It just needs not to mess with memory too much. We can reserve a region for Coreboot say at 1MB-2MB (plus usual reserved regions like 0-4K and 960K-1MB) I dont want to take all low memory because the entry point is in real mode and the OS must have some memory bellow 1MB free. We can put _RAMBASE at 8MB and reserve region from 8MB-9MB but no other memory that is reserved should not be touched because it contains OS data ;) (so no memory test for example ;) There are parts of Coreboot which are messing too much with memory, like I described in previous mails and this must be fixed. Easiest way how to avoid non-std solutions and get instantly more memory is to escape to up 1MB of memory or to any better place ;) I dont think it is wise to have for LB only 0 - 640KB of RAM with lot of exceptions, lets use one continuous block in "higher" RAM. ACPI says, reserved blocks can start at 8MB. Problematic parts: Secondary.S - (the trampoline for AP CPUs) lapic_cpu_init.c - (it copies the Secondary.S to undefined place in low memory, and AP cpu stack is handled badly when _RAMBASE is bellow 1MB) wakeup.c (yeah mine implentation which jumps back to real mode, this must be fixed, so no data at our reserved region of 0-4KB gets overwritten) clear_init_ram.c (in AMD CAR) it just clears a lot of memory from ZERO instead of _RAMBASE to TOPK Used CAR "memory/cache" should not be a problem, because it is not written back to memory in any way. So to sum it up. If we would isolate Coreboot writes to memory to those regions: 0-4K 960K-1M _RAMBASE - TOPK (RAMBASE > 1MB) It would be nice, and just _RAMBASE - TOPK region could be reserved in memory map, because it uses Coreboot when starting after resume. This memory region could be claimed back to OS via ACPI memory hotplug, which I'm now investigating. Reserving permanently 1MB of memory when using S3 would be also possible because Asus BIOS takes 1MB at the end of memory for SMM and suspend too. I hope it is now clear how it works ;) I just avoided in my experiments all those stuff because I told linux to use 2MB-1024MB :) (my TOPK is 2048) But now I really would like to get this solved too, because suspend to RAM now works which is great leap because with proprietary BIOS I was unable to get it working :))) Rudolf -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHiTOA3J9wPJqZRNURAtDaAKC6rsRnILoF+00I25gP2PywsqbXewCfaWqF GewY+xYUH0Q2rWtVIOYP1/k= =DIfc -END PGP SIGNATURE- -- coreboot mailing list coreboot@coreboot.org http://www.coreboot.org/mailman/listinfo/coreboot
Re: [coreboot] [LinuxBIOS] Messing with CAR in a big way
Rudolf, many payloads start at 1 MB. I don't think we want to use CAR at that low address. What is the reason that we can't put all coreboot variables below 1MB? Just trying to understand s3 :-) ron -- coreboot mailing list coreboot@coreboot.org http://www.coreboot.org/mailman/listinfo/coreboot
