Re: Confirmation request about new 2.4.x. kernel limits
On Tue, Jan 09, 2001 at 08:27:49AM -0800, Tim Wright wrote: > On Tue, Jan 09, 2001 at 02:21:56PM +0200, Matti Aarnio wrote: > [...] > > On the other hand, Alpha systems and SPARC systems have IOMMU hardware, > > and we do support that (to some extent), but 32-bit intel world doesn't > > have similar things. > > you are correct in saying that ia32 systems don't have IOMMU hardware, Well, there is: AGPGART does something like this. I don't know, if it support more than 32bits address space, though. > but it's unfortunate that we don't support 64-bit PCI bus master cards, Indeed. Regards, -- Kurt Garloff <[EMAIL PROTECTED]> Eindhoven, NL GPG key: See mail header, key servers Linux kernel development SuSE GmbH, Nuernberg, FRG SCSI, Security PGP signature
RE: Confirmation request about new 2.4.x. kernel limits
> On Tue, Jan 09, 2001 at 08:27:49AM -0800, Tim Wright wrote: > > you are correct in saying that ia32 systems don't have IOMMU hardware, > but > > it's unfortunate that we don't support 64-bit PCI bus master cards, > since > > they're inexpensive and fairly common now. For instance, the Qlogic ISP > SCSI > > cards can do 64-bit addressing, as can many others. Has anybody taken a > look > > at enabling this ? [Venkatesh Ramamurthy] Our AMI RAID controller and MegaRAID driver ( part of kernel) supports full blown 64 bit and we have successfully tested with more than 4 GB RAM under IA32 (uses bounce buffer) and IA64 systems. We are seeing a dramatic performance drop on IA32 with 8GB ram as it has to do bounce buffering. I think we need this support to give our dear old linux a lead compared to other OS'es. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Confirmation request about new 2.4.x. kernel limits
On Tue, Jan 09, 2001 at 12:35:02PM -0500, Venkatesh Ramamurthy wrote: > > > Problem is that it needs a driver interface change and cooperation from > > the > > drivers. > [Venkatesh Ramamurthy] Atleast the spec for this new interface, > that the driver has to support be prepared? Once this is done we can port > driver by driver to this new standard. AFAIK there is no spec yet. Just supporting 64bit DMA on 32bit hosts would probably only minor changes (like pushing the dma_mask flag a bit more out so that it is visible by scsi or ll_rw_blk), but 2.5 will probably also see more extensive changes for block drivers, like moving them to kiovecs. Your input would be probably welcome. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
RE: Confirmation request about new 2.4.x. kernel limits
> Problem is that it needs a driver interface change and cooperation from > the > drivers. [Venkatesh Ramamurthy] Atleast the spec for this new interface, that the driver has to support be prepared? Once this is done we can port driver by driver to this new standard. > -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Confirmation request about new 2.4.x. kernel limits
On Tue, Jan 09, 2001 at 05:44:46PM +0100, Andi Kleen wrote: > On Tue, Jan 09, 2001 at 08:27:49AM -0800, Tim Wright wrote: > > you are correct in saying that ia32 systems don't have IOMMU hardware, but > > it's unfortunate that we don't support 64-bit PCI bus master cards, since > > they're inexpensive and fairly common now. For instance, the Qlogic ISP SCSI > > cards can do 64-bit addressing, as can many others. Has anybody taken a look > > at enabling this ? > > Problem is that it needs a driver interface change and cooperation from the > drivers. > > -Andi I thought as much. Sounds like 2.5 material really, although it might be interesting to see if I can make a patch to play with. Regards, Tim -- Tim Wright - [EMAIL PROTECTED] or [EMAIL PROTECTED] or [EMAIL PROTECTED] IBM Linux Technology Center, Beaverton, Oregon "Nobody ever said I was charming, they said "Rimmer, you're a git!"" RD VI - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Confirmation request about new 2.4.x. kernel limits
On Tue, Jan 09, 2001 at 08:27:49AM -0800, Tim Wright wrote: > you are correct in saying that ia32 systems don't have IOMMU hardware, but > it's unfortunate that we don't support 64-bit PCI bus master cards, since > they're inexpensive and fairly common now. For instance, the Qlogic ISP SCSI > cards can do 64-bit addressing, as can many others. Has anybody taken a look > at enabling this ? Problem is that it needs a driver interface change and cooperation from the drivers. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Confirmation request about new 2.4.x. kernel limits
On Tue, Jan 09, 2001 at 02:21:56PM +0200, Matti Aarnio wrote: [...] > > For IO on usual systems you have 32 bit address space PCI busmasters, > so those can access only the lowest 4GB of address space, and to have > a block of data in upper area, it needs to be "bounced", that is, CPU > must copy it. Linux 2.4.0 system doesn't support 64-bit PCI addresses > at 32-bit systems (not at 64-bit Alpha either, I recall.) > On the other hand, Alpha systems and SPARC systems have IOMMU hardware, > and we do support that (to some extent), but 32-bit intel world doesn't > have similar things. > Hi Matti, you are correct in saying that ia32 systems don't have IOMMU hardware, but it's unfortunate that we don't support 64-bit PCI bus master cards, since they're inexpensive and fairly common now. For instance, the Qlogic ISP SCSI cards can do 64-bit addressing, as can many others. Has anybody taken a look at enabling this ? Tim -- Tim Wright - [EMAIL PROTECTED] or [EMAIL PROTECTED] or [EMAIL PROTECTED] IBM Linux Technology Center, Beaverton, Oregon "Nobody ever said I was charming, they said "Rimmer, you're a git!"" RD VI - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Confirmation request about new 2.4.x. kernel limits
On Tue, Jan 09, 2001 at 10:15:34AM -0500, Venkatesh Ramamurthy wrote: > > Any memory over 1GB is bounce-buffered, but we don't use that memory > > for anything other than process data pages or file cache, so only > > swapping and disk IO to regular files gets the extra copy. In > > particular, things like network buffers are still all kept in the low > > 1GB so never need to be buffered. > [Venkatesh Ramamurthy] If anything over 1GB is bounce buffered than > what is the purpose of setting the pci_dev->dma_mask field. On a IA32 > system we set it to 32 1's and IA64 to 64 1's - Venkat On 2.4 on 32bit hosts it has no direct effect because the driver interface has not been revised. It's only useful on 64bit atm. 2.5 will hopefully move the bounce test into your driver. Currently it tests PageHighMem() which is bogus (and gives the 1GB boundary), because it should test the physical address against the dma_mask. On 64bit PageHighMem always returns false, so the bounce buffer is never create in the upper level and the final decision is down in your driver. Currently create_bounce() in the block layer is at a very unfortunate place on the top half of ll_rw_blk, because there it has not even a chance to know about the final device (e.g. when LVM or MD come into play) I suspect someone will quickly do a limited 2.4 patch that moves create_bounce a bit down for selected devices on 32bit hosts -- it is a very obvious optimization. It probably also could be done with minor changes. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
RE: Confirmation request about new 2.4.x. kernel limits
> Any memory over 1GB is bounce-buffered, but we don't use that memory > for anything other than process data pages or file cache, so only > swapping and disk IO to regular files gets the extra copy. In > particular, things like network buffers are still all kept in the low > 1GB so never need to be buffered. [Venkatesh Ramamurthy] If anything over 1GB is bounce buffered than what is the purpose of setting the pci_dev->dma_mask field. On a IA32 system we set it to 32 1's and IA64 to 64 1's - Venkat > > - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Confirmation request about new 2.4.x. kernel limits
Hi, On Mon, Jan 08, 2001 at 11:11:05PM -0500, Venkatesh Ramamurthy wrote: > > > Max. RAM size:64 GB (any slowness > accessing RAM over 4 GB > * with 32 bit machines ?) > Imore than 4GB in RAM is bounce buffered, so there is performance > penalty as the data have to be copied into the 4GB RAM area Any memory over 1GB is bounce-buffered, but we don't use that memory for anything other than process data pages or file cache, so only swapping and disk IO to regular files gets the extra copy. In particular, things like network buffers are still all kept in the low 1GB so never need to be buffered. --Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Confirmation request about new 2.4.x. kernel limits
Hi, On Fri, Jan 05, 2001 at 11:46:04PM +0100, Pavel Machek wrote: > > > Max. file size: 1 TB(?) > > Max. file system size: 2 TB(?) > > Again, maybe on i386 with ext2. Actually, the 2TB limit affects all architectures, as we assume that block indexes fit into 32 bits. Blocks are passed around as unsigned longs in some cases, but even on 64-bit machines that doesn't help us as the limit still persists in the filesystem (32-bit block numbers) and device drivers (ints and 4-byte sector numbers used when generating SCSI commands). Auditing the whole driver path to allow 64-bit block numbers, and adding the logic to generate the 5th sector address byte in the scsi command when we're doing 10-byte commands, are all possible extensions for 2.5. For now, though, the 2TB device limit is with us for all architectures and all filesystems on 2.4. --Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Confirmation request about new 2.4.x. kernel limits
On Mon, Jan 08, 2001 at 11:11:05PM -0500, Venkatesh Ramamurthy wrote: > > > Max. RAM size: 64 GB (any slowness accessing RAM over 4 GB > > with 32 bit machines ?) > more than 4GB in RAM is bounce buffered, so there is performance > penalty as the data have to be copied into the 4GB RAM area Actual memory limit is lower, your run-of-the-mill Pentium-PAE36 capable system has PCI bus(es) for IO, and address space for that/those need to stay in area below 4G for bootup to access any devices, thus very likely your system doesn't have more than, say, 3 GB of RAM below 4G. Pick your processors. You need XEONs to have L1/L2 cacheing on memory above the 4 GB address (PAE36 mapped physical addresses.) For IO on usual systems you have 32 bit address space PCI busmasters, so those can access only the lowest 4GB of address space, and to have a block of data in upper area, it needs to be "bounced", that is, CPU must copy it. Linux 2.4.0 system doesn't support 64-bit PCI addresses at 32-bit systems (not at 64-bit Alpha either, I recall.) On the other hand, Alpha systems and SPARC systems have IOMMU hardware, and we do support that (to some extent), but 32-bit intel world doesn't have similar things. For userspace, if parts of userspace are physically mapped above 4G, it might not be very harmfull at all -- presuming you have XEONs which cache the memory accesses there also. The libc and similar multiply shared objects might as well reside in high memory. Userspace process doesn't see, after all, where each page resides physically. /Matti Aarnio - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
RE: Confirmation request about new 2.4.x. kernel limits
> Max. RAM size:64 GB (any slowness accessing RAM over 4 GB * with 32 bit machines ?) Imore than 4GB in RAM is bounce buffered, so there is performance penalty as the data have to be copied into the 4GB RAM area - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Confirmation request about new 2.4.x. kernel limits
Hi! > Hi, I would like to know whether following limits are right for kernel > 2.4.x: > > Max. N. of CPU: 32 (SMP) > Max. CPU speed: > 2 Ghz (up to ?) > Max. RAM size:64 GB (any slowness accessing RAM over 4 GB >with 32 bit machines ?) 64GB on i386. I believe you can get few terabytes with ultrasparc. > Max. file size: 1 TB(?) > Max. file system size:2 TB(?) Again, maybe on i386 with ext2. Pavel -- I'm [EMAIL PROTECTED] "In my country we have almost anarchy and I don't care." Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Confirmation request about new 2.4.x. kernel limits
> Hi, I would like to know whether following limits are right for kernel > 2.4.x: > > Max. N. of CPU: 32 (SMP) Max CPUs is 64 on 64 bit architectures (well you have to change NR_CPUS). I am told larger than 32 cpu ultrasparcs have booted linux already. Anton - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Confirmation request about new 2.4.x. kernel limits
On Thu, 4 Jan 2001, A.D.F. wrote: > Max. RAM size:64 GB (any slowness accessing RAM over 4 GB >with 32 bit machines ?) realistic benchmarks (unixbench) will show about 3%-6% performance degradation with use of PAE. Note that this is not "accessing RAM over 4G" but (what you probably meant) "accessing any RAM in a machine with over 4G of RAM" or even "accessing any RAM in a machine with less than 4G or RAM but running kernel capable of accessing >4G". If you really meant "accessing RAM over 4G" then you are probably talking about 36bit MTRR support which is present in recent 2.4.x kernels and works very nicely! You can construct artificial benchmarks that will show 10% performance degradation. I haven't compared this with other PAE implementation, e.g. that of UnixWare 7.1.1+ -- this would be a very useful exercise as I vaguely remember some people claiming that Linux PAE implementation is not ideal (if it is true then it ought to be made ideal to be inline with the rest of the kernel). Regards, Tigran - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
