Hi, The original thread was posted in the device drivers and zfs discussion.
http://www.opensolaris.org/jive/thread.jspa?messageID=162462𧪞 However the problem seems to only occur under the xen dom0: I boot the kernel without the hypervisor, the problem completely disappear. Upgrading the raid card firmware didn't help, turning off write-back didn't help (actually made it worse.) This is a 4 core amd 64 machine with 8 gig of ram. I remember there was an old discussion on xen users related to dma on machine with big memory configuration. Here is the description of the problem. Just as I create a ZFS pool and copy the root partition to it.... the performance seems to be really good then suddenly the system hangs all my sesssions and displays on the console: Oct 10 00:23:28 sunrise arcmsr: WARNING: arcmsr0: dma map got 'no resources' Oct 10 00:23:28 sunrise arcmsr: WARNING: arcmsr0: dma allocate fail Oct 10 00:23:28 sunrise arcmsr: WARNING: arcmsr0: dma allocate fail free scsi hba AR1100 was running in JBOD mode and write-back caching under Solaris Express Xen Drop b66 07/07. Never had problem this setup under linux under heavy load. ARECA SATA-SAS RAID Host Adapter Driver(i386) 1.20.00.13,REV=2006.08.14 Updated to latest firmware 1.43-70417 ... same problem.. WARNING: arcmsr0: dma map got 'no resources' WARNING: arcmsr0: dma allocate fail WARNING: arcmsr0: dma allocate fail free scsi hba pkt WARNING: arcmsr0: dma map got 'no resources' WARNING: arcmsr0: dma allocate fail WARNING: a SunOS sunrise 5.11 xen-nv66-2007-06-24 i86pc i386 i86pc prtconf -pv System Configuration: Sun Microsystems i86pc Memory size: 8192 Megabytes System Peripherals (PROM Nodes): Node 0x000001 bios-boot-device: '81' stdout: 00000000 name: 'i86pc' Node 0x000002 existing: 00d5d000.00000000.02368001.00000000 name: 'ramdisk' Node 0x000003 bus-type: 'isa' device_type: 'isa' name: 'isa' Node 0x000004 device_type: 'pci' reg: 00000000.00000000.00000000 #size-cells: 00000002 #address-cells: 00000003 name: 'pci' Node 0x000005 reg: 00000800.00000000.00000000.00000000.00000000 compatible: 'pci1166,36.0' + 'pci1166,36' + 'pciclass,060400' + 'pciclass,0604' model: 'PCI-PCI bridge' ranges: 82000000.00000000.ff400000.82000000.00000000.ff400000.00000000.00100000.c2000000.00000000.f6300000.c2000000.00000000.f6300000.00000000.00800000 bus-range: 00000001.00000003 #size-cells: 00000002 #address-cells: 00000003 device_type: 'pci' power-consumption: 00000001.00000001 devsel-speed: 00000000 unit-address: '1' class-code: 00060400 revision-id: 00000000 vendor-id: 00001166 device-id: 00000036 pcix-capid-pointer: 00000090 name: 'pci1166,36' Node 0x000018 slot-names: 00000008.746f6c53.00000031 reg: 00016800.00000000.00000000.00000000.00000000 compatible: 'pci1166,104.c0' + 'pci1166,104' + 'pciclass,060400' + 'pciclass,0604' model: 'PCI-PCI bridge' ranges: 82000000.00000000.ff400000.82000000.00000000.ff400000.00000000.00100000.c2000000.00000000.f6300000.c2000000.00000000.f6300000.00000000.00800000 bus-range: 00000002.00000003 #size-cells: 00000002 #address-cells: 00000003 device_type: 'pci' power-consumption: 00000001.00000001 66mhz-capable: devsel-speed: 00000001 unit-address: 'd' class-code: 00060400 revision-id: 000000c0 vendor-id: 00001166 device-id: 00000104 pcix-capid-pointer: 00000090 name: 'pci1166,104' Node 0x000019 reg: 00021800.00000000.00000000.00000000.00000000 compatible: 'pci8086,335.a' + 'pci8086,335' + 'pciclass,060400' + 'pciclass,0604' model: 'PCI-PCI bridge' ranges: 82000000.00000000.ff400000.82000000.00000000.ff400000.00000000.00100000.c2000000.00000000.f6300000.c2000000.00000000.f6300000.00000000.00800000 bus-range: 00000003.00000003 #size-cells: 00000002 #address-cells: 00000003 device_type: 'pci' power-consumption: 00000001.00000001 66mhz-capable: fast-back-to-back: devsel-speed: 00000001 unit-address: '3' class-code: 00060400 revision-id: 0000000a vendor-id: 00008086 device-id: 00000335 pcix-capid-pointer: 000000f0 name: 'pci8086,335' Node 0x00001a assigned-addresses: 82037010.00000000.ff4ff000.00000000.00001000.c2037018.00000000.f6400000.00000000.00400000 reg: 00037000.00000000.00000000.00000000.00000000.02037010.00000000.00000000.00000000.00001000.42037018.00000000.00000000.00000000.00400000 compatible: 'pci17d3,1110.17d3.1110.0' + 'pci17d3,1110.17d3.1110' + 'pci17d3,1110' + 'pci17d3,1110.0' + 'pci17d3,1110' + 'pciclass,010400' + 'pciclass,0104' model: 'RAID controller' power-consumption: 00000001.00000001 66mhz-capable: devsel-speed: 00000001 interrupts: 00000001 max-latency: 00000000 min-grant: 00000080 subsystem-vendor-id: 000017d3 subsystem-id: 00001110 unit-address: 'e' class-code: 00010400 revision-id: 00000000 vendor-id: 000017d3 device-id: 00001110 pcix-capid-pointer: 000000e0 name: 'pci17d3,1110' Node 0x000006 reg: 00001000.00000000.00000000.00000000.00000000 compatible: 'pci1166,205.1166.201.0' + 'pci1166,205.1166.201' + 'pci1166,201' + 'pci1166,205.0' + 'pci1166,205' + 'pciclass,060000' + 'pciclass,0600' model: 'Host bridge' power-consumption: 00000001.00000001 devsel-speed: 00000001 max-latency: 00000000 min-grant: 00000000 subsystem-vendor-id: 00001166 subsystem-id: 00000201 unit-address: '2' class-code: 00060000 revision-id: 00000000 vendor-id: 00001166 device-id: 00000205 name: 'pci1166,201' Node 0x000007 #size-cells: 00000000 #address-cells: 00000001 device_type: 'pci-ide' assigned-addresses: 81001110.00000000.000001f0.00000000.00000008.81001114.00000000.000003f6.00000000.00000001.81001118.00000000.00000170.00000000.00000008.8100111c.00000000.00000376.00000000.00000001.81001120.00000000.0000ffa0.00000000.00000010 reg: 00001100.00000000.00000000.00000000.00000000.81001110.00000000.000001f0.00000000.00000008.81001114.00000000.000003f6.00000000.00000001.81001118.00000000.00000170.00000000.00000008.8100111c.00000000.00000376.00000000.00000001.01001120.00000000.00000000.00000000.00000010 compatible: 'pci1166,214.1166.214.0' + 'pci1166,214.1166.214' + 'pci1166,214' + 'pci1166,214.0' + 'pci1166,214' + 'pciclass,01018a' + 'pciclass,0101' model: 'IDE controller' power-consumption: 00000001.00000001 devsel-speed: 00000001 max-latency: 00000000 min-grant: 00000000 subsystem-vendor-id: 00001166 subsystem-id: 00000214 unit-address: '2,1' class-code: 0001018a revision-id: 00000000 vendor-id: 00001166 device-id: 00000214 name: 'pci-ide' Node 0x000008 reg: 00000000 name: 'ide' Node 0x000009 reg: 00000001 name: 'ide' Node 0x00000a reg: 00001200.00000000.00000000.00000000.00000000 compatible: 'pci1166,234.1166.230.0' + 'pci1166,234.1166.230' + 'pci1166,230' + 'pci1166,234.0' + 'pci1166,234' + 'pciclass,060100' + 'pciclass,0601' model: 'ISA bridge' power-consumption: 00000001.00000001 devsel-speed: 00000001 max-latency: 00000000 min-grant: 00000000 subsystem-vendor-id: 00001166 subsystem-id: 00000230 unit-address: '2,2' class-code: 00060100 revision-id: 00000000 vendor-id: 00001166 device-id: 00000234 name: 'pci1166,230' Node 0x00000b assigned-addresses: 82001810.00000000.ff6b4000.00000000.00001000.81001814.00000000.0000e000.00000000.00000100 reg: 00001800.00000000.00000000.00000000.00000000.02001810.00000000.00000000.00000000.00001000.01001814.00000000.00000000.00000000.00000100 compatible: 'pci1166,223.1166.223.1' + 'pci1166,223.1166.223' + 'pci1166,223' + 'pci1166,223.1' + 'pci1166,223' + 'pciclass,0c0310' + 'pciclass,0c03' model: 'Universal Serial Bus OHCI compliant' power-consumption: 00000001.00000001 66mhz-capable: fast-back-to-back: devsel-speed: 00000001 interrupts: 00000001 max-latency: 00000000 min-grant: 00000000 subsystem-vendor-id: 00001166 subsystem-id: 00000223 unit-address: '3' class-code: 000c0310 revision-id: 00000001 vendor-id: 00001166 device-id: 00000223 name: 'pci1166,223' Node 0x00000c assigned-addresses: 82001910.00000000.ff6b5000.00000000.00001000.81001914.00000000.0000e400.00000000.00000100 reg: 00001900.00000000.00000000.00000000.00000000.02001910.00000000.00000000.00000000.00001000.01001914.00000000.00000000.00000000.00000100 compatible: 'pci1166,223.1166.223.1' + 'pci1166,223.1166.223' + 'pci1166,223' + 'pci1166,223.1' + 'pci1166,223' + 'pciclass,0c0310' + 'pciclass,0c03' model: 'Universal Serial Bus OHCI compliant' power-consumption: 00000001.00000001 66mhz-capable: fast-back-to-back: devsel-speed: 00000001 interrupts: 00000001 max-latency: 00000000 min-grant: 00000000 subsystem-vendor-id: 00001166 subsystem-id: 00000223 unit-address: '3,1' class-code: 000c0310 revision-id: 00000001 vendor-id: 00001166 device-id: 00000223 name: 'pci1166,223' Node 0x00000d assigned-addresses: 82001a10.00000000.ff6b6000.00000000.00001000.81001a14.00000000.0000e800.00000000.00000100 reg: 00001a00.00000000.00000000.00000000.00000000.02001a10.00000000.00000000.00000000.00001000.01001a14.00000000.00000000.00000000.00000100 compatible: 'pci1166,223.1166.223.1' + 'pci1166,223.1166.223' + 'pci1166,223' + 'pci1166,223.1' + 'pci1166,223' + 'pciclass,0c0320' + 'pciclass,0c03' model: 'Universal Serial Bus EHCI compliant' power-consumption: 00000001.00000001 66mhz-capable: fast-back-to-back: devsel-speed: 00000001 interrupts: 00000001 max-latency: 00000000 min-grant: 00000000 subsystem-vendor-id: 00001166 subsystem-id: 00000223 unit-address: '3,2' class-code: 000c0320 revision-id: 00000001 vendor-id: 00001166 device-id: 00000223 name: 'pci1166,223' Node 0x00000e assigned-addresses: 82002010.00000000.ff680000.00000000.00020000.82002014.00000000.ff660000.00000000.00020000.81002018.00000000.0000dc00.00000000.00000040 reg: 00002000.00000000.00000000.00000000.00000000.02002010.00000000.00000000.00000000.00020000.02002014.00000000.00000000.00000000.00020000.01002018.00000000.00000000.00000000.00000040 compatible: 'pci8086,1076.10f1.3870.5' + 'pci8086,1076.10f1.3870' + 'pci10f1,3870' + 'pci8086,1076.5' + 'pci8086,1076' + 'pciclass,020000' + 'pciclass,0200' model: 'Ethernet controller' power-consumption: 00000001.00000001 66mhz-capable: devsel-speed: 00000001 interrupts: 00000001 max-latency: 00000000 min-grant: 000000ff subsystem-vendor-id: 000010f1 subsystem-id: 00003870 unit-address: '4' class-code: 00020000 revision-id: 00000005 vendor-id: 00008086 device-id: 00001076 pcix-capid-pointer: 000000e4 name: 'pci10f1,3870' Node 0x00000f video-adapter-type: 'svga' display-type: 'color' assigned-addresses: c2003010.00000000.f8000000.00000000.04000000.82003014.00000000.ff6c0000.00000000.00040000.81003018.00000000.0000ec00.00000000.00000080.a1003000.00000000.000003b0.00000000.0000000c.a1003000.00000000.000003c0.00000000.00000020.82003000.00000000.000a0000.00000000.00020000 reg: 00003000.00000000.00000000.00000000.00000000.42003010.00000000.00000000.00000000.04000000.02003014.00000000.00000000.00000000.00040000.01003018.00000000.00000000.00000000.00000080.a1003000.00000000.000003b0.00000000.0000000c.a1003000.00000000.000003c0.00000000.00000020.82003000.00000000.000a0000.00000000.00020000 compatible: 'pci18ca,20.18ca.20.0' + 'pci18ca,20.18ca.20' + 'pci18ca,20' + 'pci18ca,20.0' + 'pci18ca,20' + 'pciclass,030000' + 'pciclass,0300' model: 'VGA compatible controller' power-consumption: 00000001.00000001 66mhz-capable: devsel-speed: 00000001 max-latency: 00000000 min-grant: 00000000 subsystem-vendor-id: 000018ca subsystem-id: 00000020 device_type: 'display' unit-address: '6' class-code: 00030000 revision-id: 00000000 vendor-id: 000018ca device-id: 00000020 name: 'display' Node 0x000010 reg: 0000c000.00000000.00000000.00000000.00000000 compatible: 'pci1022,1100.0' + 'pci1022,1100' + 'pciclass,060000' + 'pciclass,0600' model: 'Host bridge' power-consumption: 00000001.00000001 devsel-speed: 00000000 max-latency: 00000000 min-grant: 00000000 unit-address: '18' class-code: 00060000 revision-id: 00000000 vendor-id: 00001022 device-id: 00001100 name: 'pci1022,1100' Node 0x000011 reg: 0000c100.00000000.00000000.00000000.00000000 compatible: 'pci1022,1101.0' + 'pci1022,1101' + 'pciclass,060000' + 'pciclass,0600' model: 'Host bridge' power-consumption: 00000001.00000001 devsel-speed: 00000000 max-latency: 00000000 min-grant: 00000000 unit-address: '18,1' class-code: 00060000 revision-id: 00000000 vendor-id: 00001022 device-id: 00001101 name: 'pci1022,1101' Node 0x000012 reg: 0000c200.00000000.00000000.00000000.00000000 compatible: 'pci1022,1102.0' + 'pci1022,1102' + 'pciclass,060000' + 'pciclass,0600' model: 'Host bridge' power-consumption: 00000001.00000001 devsel-speed: 00000000 max-latency: 00000000 min-grant: 00000000 unit-address: '18,2' class-code: 00060000 revision-id: 00000000 vendor-id: 00001022 device-id: 00001102 name: 'pci1022,1102' Node 0x000013 reg: 0000c300.00000000.00000000.00000000.00000000 compatible: 'pci1022,1103.0' + 'pci1022,1103' + 'pciclass,060000' + 'pciclass,0600' model: 'Host bridge' power-consumption: 00000001.00000001 devsel-speed: 00000000 max-latency: 00000000 min-grant: 00000000 unit-address: '18,3' class-code: 00060000 revision-id: 00000000 vendor-id: 00001022 device-id: 00001103 name: 'pci1022,1103' Node 0x000014 reg: 0000c800.00000000.00000000.00000000.00000000 compatible: 'pci1022,1100.0' + 'pci1022,1100' + 'pciclass,060000' + 'pciclass,0600' model: 'Host bridge' power-consumption: 00000001.00000001 devsel-speed: 00000000 max-latency: 00000000 min-grant: 00000000 unit-address: '19' class-code: 00060000 revision-id: 00000000 vendor-id: 00001022 device-id: 00001100 name: 'pci1022,1100' Node 0x000015 reg: 0000c900.00000000.00000000.00000000.00000000 compatible: 'pci1022,1101.0' + 'pci1022,1101' + 'pciclass,060000' + 'pciclass,0600' model: 'Host bridge' power-consumption: 00000001.00000001 devsel-speed: 00000000 max-latency: 00000000 min-grant: 00000000 unit-address: '19,1' class-code: 00060000 revision-id: 00000000 vendor-id: 00001022 device-id: 00001101 name: 'pci1022,1101' Node 0x000016 reg: 0000ca00.00000000.00000000.00000000.00000000 compatible: 'pci1022,1102.0' + 'pci1022,1102' + 'pciclass,060000' + 'pciclass,0600' model: 'Host bridge' power-consumption: 00000001.00000001 devsel-speed: 00000000 max-latency: 00000000 min-grant: 00000000 unit-address: '19,2' class-code: 00060000 revision-id: 00000000 vendor-id: 00001022 device-id: 00001102 name: 'pci1022,1102' Node 0x000017 reg: 0000cb00.00000000.00000000.00000000.00000000 compatible: 'pci1022,1103.0' + 'pci1022,1103' + 'pciclass,060000' + 'pciclass,0600' model: 'Host bridge' power-consumption: 00000001.00000001 devsel-speed: 00000000 max-latency: 00000000 min-grant: 00000000 unit-address: '19,3' class-code: 00060000 revision-id: 00000000 vendor-id: 00001022 device-id: 00001103 name: 'pci1022,1103' This message posted from opensolaris.org _______________________________________________ xen-discuss mailing list [email protected]
