Mark Knecht posted <[EMAIL PROTECTED]>, excerpted below, on Sat, 15 Oct 2005 11:18:12 -0700:
> Is anyone else seeign these troubling messages from tie to time? > > mtrr: base(0xe8020000) is not aligned on a size(0x400000) boundary > mtrr: type mismatch for e8000000,8000000 old: write-back new: write-combining > mtrr: type mismatch for e8000000,8000000 old: write-back new: write-combining > mtrr: type mismatch for e8000000,8000000 old: write-back new: write-combining > mtrr: base(0xe8020000) is not aligned on a size(0x400000) boundary > mtrr: type mismatch for e8000000,8000000 old: write-back new: write-combining > mtrr: base(0xe8020000) is not aligned on a size(0x400000) boundary > mtrr: type mismatch for e8000000,8000000 old: write-back new: write-combining > mtrr: base(0xe8020000) is not aligned on a size(0x400000) boundary > mtrr: type mismatch for e8000000,8000000 old: write-back new: write-combining > > Is there somethign I should be doing to fix this? You may well know more about this than I do, but on the off chance this may be new to you and for others... (and because I'm googling and learning a bit in the process myself... =8^) mtrr=memory type range (or region) register. This is definitely kernel domain we are talking about here, often video drivers (so xorg related as well). Here's a link to an old but decently "Engish" explanation of what MTRRs are that doesn't get too technical, but gives you some idea of not only what the theory does, but the effects on real-world performance. (It's talking PentiumPro/PII CPUs and kernel 2.2.0 or later... I /said/ it was old! =8^) http://www.meduna.org/txt_mtrr_en.html Paraphrasing from the above link... Basically, the MTRRs determine the behavior of cache vs regular memory on memory-write, for various memory regions/ranges. * Write-thru means the write is to cache and main memory together, slower than Write-back or Write-combining below but most reliable as any DMAs from main memory are guaranteed to be up-to-date. * Write-back is far faster, allowing the CPU to write to cache only then go about its business, with the update to main memory happening as memory bandwidth permits. The catch is that cache (play on words there! <g>) and main memory can be out of sync momentarily, which could mean either longer waits for the update to happen when it /has/ to happen (if everything is working right and the out-of-date-data is caught and a wait forced until it's updated to valid data when needed), or in the worst-case scenario, if something's not quite working right, glitches, instability and crashes because old and invalid data was used where updated data was expected. * Write-combining is sort of in-between the two above choices. The data is allowed to sit in cache without updating main memory only a comparatively short period, in ordered to allow the possibility of combining several smaller writes into a larger, single, more efficient write. (More efficient because each write has a set amount of overhead in setup and take-down time and data. Thus, combining 8 128-byte transactions into a single 1KB transaction means 1/8 the overhead, thus more effective payload bandwidth, at a cost of more latency, due to waiting for several transactions to accumulate, provided of course they come in before the expiry time forces what's there to be written even if it's not yet a full size transfer.) * There's also uncachable, which turns off caching for reads as well as writes. This will be /very/ slow. Where graphics gets involved is that these MTRRs are often used to program access to video memory over the AGP or whatever bus. The link above lists some of the (then) xfree86 operations that MTRR settings affect and by how much. Another link with some interesting info on the kernel config option (CONFIG_MTRR) and the userland interface to MTRRs (/proc/mtrr) once enabled. The two paragraphs below are excerpted: http://developer.osdl.org/dev/robustmutexes/src/fusyn.hg/Documentation/mtrr.txt The CONFIG_MTRR option creates a /proc/mtrr file which may be used to manipulate your MTRRs. Typically the X server should use this. This should have a reasonably generic interface so that similar control registers on other processors can be easily supported. There are two interfaces to /proc/mtrr: one is an ASCII interface which allows you to read and write. The other is an ioctl() interface. The ASCII interface is meant for administration. The ioctl() interface is meant for C programs (i.e. the X server). The interfaces are described below, with sample commands and C code. </quote> Finally, this, on the mtrr_add command from kernelnewbies: http://kernelnewbies.org/documents/kdoc/kernel-api/r7666.html Memory type region registers control the caching on newer Intel and non Intel processors. This function allows drivers to request an MTRR is added. The details and hardware specifics of each processor's implementation are hidden from the caller, but nevertheless the caller should expect to need to provide a power of two size on an equivalent power of two boundary. If the region cannot be added either because all regions are in use or the CPU cannot support it a negative value is returned. On success the register number for this entry is returned, but should be treated as a cookie only. On a multiprocessor machine the changes are made to all processors. This is required on x86 by the Intel processors. The available types are MTRR_TYPE_UNCACHABLE - No caching MTRR_TYPE_WRBACK - Write data back in bursts whenever MTRR_TYPE_WRCOMB - Write data back soon but allow bursts MTRR_TYPE_WRTHROUGH - Cache reads but not writes </quote> That last contains a mention of boundaries ("power of two size on an equivalent power of two boundary") that appears to pertain to your problem. So... now we know a bit about what MTRRs actually do (control the interaction between cache and a specified portion of memory for write transactions), what they are most often adjusted for (to increase graphics performance, by changing the way writes to graphics memory are cached), and can make a bit of sense out of the messages (the size doesn't match the required base address for the MTRR, something's trying to change the caching method, but using the wrong address and the adjustment is therefore getting a type mismatch). How does that translate into something you can do to fix it? Well, first, you can actually go take a look at /proc/mtrr (don't try to write anything to it, unless you are sure you know what you are doing, but reading it should be fine), and see if you can figure out which entry it's supposed to be changing, if there's one close to that address at all, or if not, what needs created. Beyond that, it depends on what is actually using the MTRR. It's probably your video card driver, but it could be something else. You don't say which card you have, but from the googling I did to find the above, I see that ATI's proprietary drivers at least, have posts about changing MTRRs to increase performance. Whatever your video card driver is, that's probably (but not for certain) what's causing the log messages. Therefore, take a look at /var/log/Xorg.0.log, and see if you can match up any possible MTRR messages listed there. Next, take a look at the driver documentation and your xorg.conf file and boot loader config, and see what sort of adjustments you might need to make. Of course, if it's /not/ video driver related, you'll likely have to figure out what else is accessing the MTRRs and how to reconfigure it correctly. Taking a wild guess, I'd say check anything that's likely to be using DMA, thus, stuff like NICs or storage devices, and their drivers. Oh, MTRRs are also used in connection with mapping around the PCI device memory hole just below 4GB, if you have 4G or more of memory. Hmm... Now I'm beginning to integrate what I just learned here, with some other stuff I just read, with stuff I knew before, and a real-time look at /proc/mtrr on my system... and things are beginning to "click". You get to see my understanding developing as I write this! <g> >From an Opteron BIOS integrator's pdf @ amd... they recommend one of the variable MTRRs (there are some fixed ones covering the memory space from 640k to 1 MB as well, that must be what those common settings in BIOS must be for) be set to cover the entire physical memory range... And so I see... I have a gig of memory and see a 1024 MB MTRR set @ base-address 0x 0000 0000 (0 MB), type write-back (so read/write cacheable). That makes sense as it's telling the CPU(s, 2 in my case) that all of my main memory is fully cacheable, no special restrictions needed! Apparently, some CPUs only have two variable MTRRs, and if one is used to cover all of physical main-memory, that leaves only one available, which would be used by the video driver, so that's how Linux is normally setup. Again apparently, modern x86 (and presumably x86_64 as well) CPUs from both AMD and Intel have eight such MTRRs, so more ranges can be programed as needed. Now this is just a guess based on the fixed MTRs (memory type ranges, I'm using MTRR to reference the register, MTR to reference the range in memory it controls, here referring to the previously mentioned fixed range MTRRs/MTRs between 640k and 1M) being included in the main-mem variable range, and variable MTRs within other variable MTRs might not work quite the same (tho I expect they do), but going on that... smaller ones in larger ones can act as exceptions. That would explain the BIOS MTRR setting in many AMD64 systems for > 3.5 GB physical memory -- continuous or explicit hole for the 3.5 - 4 GB (0x e000 0000 - 0x ffff ffff) top-of-32-bit-memory-space PCI hole. The continuous option will map the entire X GB of memory as a single MTR, presumably (presumably since I've never run > 4 GB myself, and I'm making connections faster than I could look them up to verify them) with additional MTRs such as the video memory one overlaid on top of it, in the 3.5-4.0 GB PCI device space. Non-contiguous or explicit-hole would map an explicit hole to the 3.5-4 GB area. Note that this would map additional memory up above the 4 GB 32-bit barrier, out of reach of most 32-bit systems, explaining some comments on forums.amd.com I was reading. If this isn't handled correctly, even 64-bit systems won't see all their memory if it's more than 3.5 GB, since part of it will be hidden by the PCI device overlay 3.5 - 4.0 GB. Note that at least here (Tyan s2885 dual Opteron board), the BIOS actually has two related settings controlling the way > 3.5 GB of physical memory is mapped. One apparently controls the actual memory addresses, whether they skip the 3.5-4 GB range or not, the other controls the MTRRs, continuous or not. If the two don't match, it could mean all of memory is seen but not all of it is marked as cacheable, slowing access to the uncached memory range. ... Another leap of understanding... Remember those ranges need to be in power-of-2 sizes and on matching power-of-2 boundaries? I happen to have a gig of memory, an even power of two, so one MTR covers it exactly. That wouldn't work for those with 3/4 gig or 1.5 or 3 gig or some such. OTOH, I noticed a count=1 at the end of the two ranges I have mapped in my /proc/mtrr, so it would appear that could be remedied by using say 3 half-gig MTRs (count=3) stacked end-to-end, or a 1 gig and a half gig (thus two), to map that 1.5 gig area. I'm not sure if the count= would mean it's using additional MTRRs or not, tho I'd expect it would indeed mean that. Thus, non-power-of-two memory layouts will probably mean more MTRRs used to map the full memory MTR. (If there's anyone with an odd amount of memory that could verify, it'd be nice...) My second MTRR is set to cover the 128 MB range dedicated to my video card (it's got 256 MB but on 128 MB is being used, it would appear, but it's no big deal since I don't do much 3D anyway because I'm running dual head video @ 2048x1536 each). Here, it's the 128 MB beginning at exactly 3.5 MB base address (0x e000 0000), write combining. OK... 0x e000 0000 is the 3.5 GB boundary, so your 0x e802 0000 is indeed in the PCI device area... 0x 800 0000 is 128 MB, 0x 2 0000 is 8 KB, so the base address it's attempting to use is 3.5 GB + 128 MB + 8 KB. The requested size is 0x 40 0000 or 4 MB, so the closest boundary would be the 0x e800 0000 you see it trying for and getting the type mismatch. Note also that the AMD docs say disable caching (which would mean flush all pending writes) before making other MTRR changes, also. What I'd guess is happening here, then, is that these errors are occurring when you start X, and the graphics video driver tries to overlay a write-combining MTRR over top of (part of) the previously mapped write-back MTRR covering all of main-memory (which would appear to be 4GB or better, thus overlapping the 3.5-4.0 GB PCI device area). They will mean problems with the video rendering, either glitches or not as fast as it could be. (Write-back being less strict than write-combining, I'm thinking it could mean glitches, whenever the video card tries to draw memory that's not current with that in cache. However, it may not appear except under heavy 3D use, such as in games.) If that's the case, in addition to the MTRR remapping you may do either from your boot scripts or by reconfiguring your graphics card driver in xorg.conf, you may have to reset your BIOS' MTRR settings to specifically map out the pci-area hole. That's about all I have ATM... Hope this is as informative for others as it just was for me! <g> (And, if anyone's an expert on this stuff and I got it wrong, please inform me where! Better to correct any mistakes I have now while it's new than after I build a bunch more suppositions on a faulty foundation!) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman in http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html -- gentoo-amd64@gentoo.org mailing list
