Try a single processor kernel instead of the MP kernel The MP kernel is just going to use more CPU time without giving you any benefit with today's implementation
Fernando Braga [EMAIL PROTECTED] wrote: > On Nov 16, 2007 2:57 PM, Calomel <[EMAIL PROTECTED]> wrote: > > > > Doing a quick google search I see other people have also reported problems > > with the on board Broadcom BCM5708 on the dells. Can you try another > > network card like the Intel Pro/1000 MT (OpenBSD interface name: em0) ? > > > > My place of business transfers an average of 450Mbit with OpenBSD 4.1/4.2 > > with ALTQ (HFSC) without issue. CPU usage for the interrupts are around 33% > > on a amd64 2.2GHz. > > Hi Calomel, > > It took me a little longer than I expected, bit I got a dual Intel > Pro/1000 to try. Unfortunately the results weren't very different: > interrupts were at 80% when I put 90Mbps of traffic over the bridge. > > I wonder what are the major differences between our servers. Could you > send me a dmesg of your bridge ? > > fmbraga:130$ sudo pfctl -si > Status: Enabled for 0 days 00:14:19 Debug: Urgent > > State Table Total Rate > current entries 291977 > searches 28283453 32926.0/s > inserts 1779088 2071.1/s > removals 1487111 1731.2/s > Counters > match 2024938 2357.3/s > bad-offset 0 0.0/s > fragment 8 0.0/s > short 176 0.2/s > normalize 81 0.1/s > memory 0 0.0/s > bad-timestamp 0 0.0/s > congestion 0 0.0/s > ip-option 28 0.0/s > proto-cksum 0 0.0/s > state-mismatch 5530 6.4/s > state-insert 0 0.0/s > state-limit 0 0.0/s > src-limit 0 0.0/s > synproxy 0 0.0/s > fmbraga:132$ > > load averages: 0.72, 0.83, 0.55 > 06:26:40 > 33 processes: 32 idle, 1 on processor > CPU0 states: 3.4% user, 0.0% nice, 0.4% system, 83.4% interrupt, 12.8% idle > CPU1 states: 2.2% user, 0.0% nice, 0.4% system, 0.0% interrupt, 97.4% idle > CPU2 states: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > CPU3 states: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > Memory: Real: 20M/271M act/tot Free: 2963M Swap: 0K/2055M used/tot > > PID USERNAME PRI NICE SIZE RES STATE WAIT TIME CPU COMMAND > 17166 fmbraga -6 0 636K 1052K sleep/0 piperd 0:23 8.94% sed > 26875 fmbraga -6 0 456K 1028K sleep/3 piperd 0:31 3.91% sed > 29981 fmbraga 2 0 3248K 2200K sleep/3 select 0:00 0.00% sshd > 28812 fmbraga 2 0 3396K 2208K sleep/0 select 0:00 0.00% sshd > 12557 fmbraga 2 0 3416K 2152K sleep/0 select 0:00 0.00% sshd > 10314 root 2 0 3276K 2864K idle netio 0:00 0.00% sshd > 29572 _pflogd 4 0 588K 476K sleep/0 bpf 0:00 0.00% pflogd > 10264 fmbraga 2 0 924K 2072K sleep/0 poll 0:00 0.00% top > 19219 root 10 0 636K 492K sleep/0 nanosle 0:00 0.00% pfctl > 11936 root 10 0 664K 500K sleep/0 nanosle 0:00 0.00% pfctl > 30336 root 2 0 640K 1380K idle select 0:00 0.00% sshd > 19057 root 2 0 3440K 2920K idle netio 0:00 0.00% sshd > 25636 root 2 0 3668K 6248K sleep/0 select 0:00 0.00% snmpd > 29407 fmbraga 28 0 752K 1952K onproc/0 - 0:00 0.00% top > 16659 root 2 0 1320K 1752K sleep/0 select 0:00 0.00% sendmail > 24774 root 2 0 3420K 2952K idle netio 0:00 0.00% sshd > 18059 _syslogd 2 0 504K 828K sleep/0 poll 0:00 0.00% syslogd > 30791 fmbraga 18 0 624K 752K idle pause 0:00 0.00% ksh > 4597 fmbraga 18 0 776K 744K idle pause 0:00 0.00% ksh > 31505 fmbraga 2 0 3336K 2172K sleep/0 select 0:00 0.00% sshd > 4308 root 2 0 3364K 2924K idle netio 0:00 0.00% sshd > 17683 root 2 0 452K 812K idle select 0:00 0.00% inetd > 32085 fmbraga 18 0 568K 756K idle pause 0:00 0.00% ksh > 22181 fmbraga 18 0 668K 736K idle pause 0:00 0.00% ksh > 1 root 10 0 448K 384K idle wait 0:00 0.00% init > 29714 root 3 0 416K 856K idle ttyin 0:00 0.00% getty > 6541 root 3 0 284K 860K idle ttyin 0:00 0.00% getty > > new dmesg follows: > > fmbraga:134$ dmesg > OpenBSD 4.2 (GENERIC.MP) #1378: Tue Aug 28 10:48:58 MDT 2007 > [EMAIL PROTECTED]:/usr/src/sys/arch/amd64/compile/GENERIC.MP > real mem = 3488907264 (3327MB) > avail mem = 3373899776 (3217MB) > mainbus0 at root > bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xcffbc000 (62 entries) > bios0: vendor Dell Inc. version "1.3.7" date 03/26/2007 > bios0: Dell Inc. PowerEdge 1950 > acpi at mainbus0 not configured > ipmi0 at mainbus0: version 2.0 interface KCS iobase 0xca8/8 spacing 4 > mainbus0: Intel MP Specification (Version 1.4) > cpu0 at mainbus0: apid 0 (boot processor) > cpu0: Intel(R) Xeon(R) CPU E5345 @ 2.33GHz, 2327.87 MHz > cpu0: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,CX16,xTPR,NXE,LONG > cpu0: 4MB 64b/line 16-way L2 cache > cpu0: apic clock running at 332MHz > cpu1 at mainbus0: apid 2 (application processor) > cpu1: Intel(R) Xeon(R) CPU E5345 @ 2.33GHz, 2327.53 MHz > cpu1: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,CX16,xTPR,NXE,LONG > cpu1: 4MB 64b/line 16-way L2 cache > cpu2 at mainbus0: apid 3 (application processor) > cpu2: Intel(R) Xeon(R) CPU E5345 @ 2.33GHz, 2327.53 MHz > cpu2: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,CX16,xTPR,NXE,LONG > cpu2: 4MB 64b/line 16-way L2 cache > cpu3 at mainbus0: apid 1 (application processor) > cpu3: Intel(R) Xeon(R) CPU E5345 @ 2.33GHz, 2327.53 MHz > cpu3: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,CX16,xTPR,NXE,LONG > cpu3: 4MB 64b/line 16-way L2 cache > mpbios: bus 0 is type PCI > mpbios: bus 1 is type PCI > mpbios: bus 2 is type PCI > mpbios: bus 3 is type PCI > mpbios: bus 4 is type PCI > mpbios: bus 5 is type PCI > mpbios: bus 6 is type PCI > mpbios: bus 7 is type PCI > mpbios: bus 8 is type PCI > mpbios: bus 9 is type PCI > mpbios: bus 10 is type PCI > mpbios: bus 11 is type PCI > mpbios: bus 12 is type PCI > mpbios: bus 13 is type PCI > mpbios: bus 14 is type PCI > mpbios: bus 15 is type PCI > mpbios: bus 16 is type PCI > mpbios: bus 17 is type ISA > ioapic0 at mainbus0 apid 4 pa 0xfec00000, version 20, 24 pins > ioapic0: misconfigured as apic 0, remapped to apid 4 > ioapic1 at mainbus0 apid 5 pa 0xfec81000, version 20, 24 pins > ioapic1: misconfigured as apic 0, remapped to apid 5 > pci0 at mainbus0 bus 0: configuration mode 1 > pchb0 at pci0 dev 0 function 0 "Intel 5000X Host" rev 0x12 > ppb0 at pci0 dev 2 function 0 "Intel 5000 PCIE" rev 0x12 > pci1 at ppb0 bus 6 > ppb1 at pci1 dev 0 function 0 "Intel 6321ESB PCIE" rev 0x01 > pci2 at ppb1 bus 7 > ppb2 at pci2 dev 0 function 0 "Intel 6321ESB PCIE" rev 0x01 > pci3 at ppb2 bus 8 > ppb3 at pci3 dev 0 function 0 "ServerWorks PCIE-PCIX" rev 0xc3 > pci4 at ppb3 bus 9 > bnx0 at pci4 dev 0 function 0 "Broadcom BCM5708" rev 0x12: apic 4 int 16 (irq > 5) > ppb4 at pci2 dev 1 function 0 "Intel 6321ESB PCIE" rev 0x01 > pci5 at ppb4 bus 10 > ppb5 at pci1 dev 0 function 3 "Intel 6321ESB PCIE-PCIX" rev 0x01 > pci6 at ppb5 bus 11 > ppb6 at pci0 dev 3 function 0 "Intel 5000 PCIE" rev 0x12 > pci7 at ppb6 bus 1 > ppb7 at pci7 dev 0 function 0 "Intel IOP333 PCIE-PCIX" rev 0x00 > pci8 at ppb7 bus 2 > mfi0 at pci8 dev 14 function 0 "Dell PERC 5" rev 0x00: apic 5 int 14 (irq 6) > mfi0: logical drives 1, version 5.1.1-0040, 256MB RAM > scsibus0 at mfi0: 1 targets > sd0 at scsibus0 targ 0 lun 0: <DELL, PERC 5/i, 1.03> SCSI3 0/direct fixed > sd0: 152064MB, 19385 cyl, 255 head, 63 sec, 512 bytes/sec, 311427072 sec total > ppb8 at pci7 dev 0 function 2 "Intel IOP333 PCIE-PCIX" rev 0x00 > pci9 at ppb8 bus 3 > ppb9 at pci0 dev 4 function 0 "Intel 5000 PCIE" rev 0x12 > pci10 at ppb9 bus 12 > em0 at pci10 dev 0 function 0 "Intel PRO/1000 PT (82571EB)" rev 0x06: > apic 4 int 16 (irq 5), address 00:15:17:3a:65:10 > em1 at pci10 dev 0 function 1 "Intel PRO/1000 PT (82571EB)" rev 0x06: > apic 4 int 17 (irq 11), address 00:15:17:3a:65:11 > ppb10 at pci0 dev 5 function 0 "Intel 5000 PCIE" rev 0x12 > pci11 at ppb10 bus 13 > ppb11 at pci0 dev 6 function 0 "Intel 5000 PCIE" rev 0x12 > pci12 at ppb11 bus 14 > ppb12 at pci0 dev 7 function 0 "Intel 5000 PCIE" rev 0x12 > pci13 at ppb12 bus 15 > pchb1 at pci0 dev 16 function 0 "Intel 5000 Error Reporting" rev 0x12 > pchb2 at pci0 dev 16 function 1 "Intel 5000 Error Reporting" rev 0x12 > pchb3 at pci0 dev 16 function 2 "Intel 5000 Error Reporting" rev 0x12 > pchb4 at pci0 dev 17 function 0 "Intel 5000 Reserved" rev 0x12 > pchb5 at pci0 dev 19 function 0 "Intel 5000 Reserved" rev 0x12 > pchb6 at pci0 dev 21 function 0 "Intel 5000 FBD" rev 0x12 > pchb7 at pci0 dev 22 function 0 "Intel 5000 FBD" rev 0x12 > ppb13 at pci0 dev 28 function 0 "Intel 6321ESB PCIE" rev 0x09 > pci14 at ppb13 bus 4 > ppb14 at pci14 dev 0 function 0 "ServerWorks PCIE-PCIX" rev 0xc3 > pci15 at ppb14 bus 5 > bnx1 at pci15 dev 0 function 0 "Broadcom BCM5708" rev 0x12: apic 4 int > 16 (irq 5) > uhci0 at pci0 dev 29 function 0 "Intel 6321ESB USB" rev 0x09: apic 4 > int 21 (irq 11) > uhci1 at pci0 dev 29 function 1 "Intel 6321ESB USB" rev 0x09: apic 4 > int 20 (irq 10) > uhci2 at pci0 dev 29 function 2 "Intel 6321ESB USB" rev 0x09: apic 4 > int 21 (irq 11) > ehci0 at pci0 dev 29 function 7 "Intel 6321ESB USB" rev 0x09: apic 4 > int 21 (irq 11) > usb0 at ehci0: USB revision 2.0 > uhub0 at usb0: Intel EHCI root hub, rev 2.00/1.00, addr 1 > ppb15 at pci0 dev 30 function 0 "Intel 82801BA AGP" rev 0xd9 > pci16 at ppb15 bus 16 > vga1 at pci16 dev 13 function 0 "ATI ES1000" rev 0x02 > wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) > wsdisplay0: screen 1-5 added (80x25, vt100 emulation) > pcib0 at pci0 dev 31 function 0 "Intel 6321ESB LPC" rev 0x09 > pciide0 at pci0 dev 31 function 1 "Intel 6321ESB IDE" rev 0x09: DMA, > channel 0 configured to compatibility, channel 1 configured to > compatibility > atapiscsi0 at pciide0 channel 0 drive 0 > scsibus1 at atapiscsi0: 2 targets > cd0 at scsibus1 targ 0 lun 0: <HL-DT-ST, CDRW/DVD GCC4244, B101> SCSI0 > 5/cdrom removable > cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2 > pciide0: channel 1 ignored (disabled) > usb1 at uhci0: USB revision 1.0 > uhub1 at usb1: Intel UHCI root hub, rev 1.00/1.00, addr 1 > usb2 at uhci1: USB revision 1.0 > uhub2 at usb2: Intel UHCI root hub, rev 1.00/1.00, addr 1 > usb3 at uhci2: USB revision 1.0 > uhub3 at usb3: Intel UHCI root hub, rev 1.00/1.00, addr 1 > isa0 at pcib0 > isadma0 at isa0 > com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo > com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo > pckbc0 at isa0 port 0x60/5 > pckbd0 at pckbc0 (kbd slot) > pckbc0: using irq 1 for kbd slot > wskbd0 at pckbd0: console keyboard, using wsdisplay0 > pcppi0 at isa0 port 0x61 > midi0 at pcppi0: <PC speaker> > spkr0 at pcppi0 > uhub4 at uhub0 port 1: Dell product 0xa001, rev 2.00/0.00, addr 2 > uhidev0 at uhub4 port 1 configuration 1 interface 0 > uhidev0: Dell DRAC5, rev 1.10/0.00, addr 3, iclass 3/1 > ukbd0 at uhidev0: 8 modifier keys, 6 key codes > wskbd1 at ukbd0 mux 1 > wskbd1: connecting to wsdisplay0 > uhidev1 at uhub4 port 1 configuration 1 interface 1 > uhidev1: Dell DRAC5, rev 1.10/0.00, addr 3, iclass 3/1 > ums0 at uhidev1 > ums0: X report 0x0002 not supported > umass0 at uhub4 port 2 configuration 1 interface 0 > umass0: DELL INC. DRAC5 VIRTUAL MEDIA, rev 2.00/0.00, addr 4 > umass0: using SCSI over Bulk-Only > scsibus2 at umass0: 2 targets > cd1 at scsibus2 targ 1 lun 0: <Dell, Virtual CDROM, 123> SCSI0 5/cdrom > removable > umass1 at uhub4 port 2 configuration 1 interface 1 > umass1: DELL INC. DRAC5 VIRTUAL MEDIA, rev 2.00/0.00, addr 4 > umass1: using SCSI over Bulk-Only > scsibus3 at umass1: 2 targets > sd1 at scsibus3 targ 1 lun 0: <Dell, Virtual Floppy, 123> SCSI0 > 0/direct removable > sd1: drive offline > uhub5 at uhub0 port 5: Cypress Semiconductor USB2 Hub, rev 2.00/0.0b, addr 5 > uhidev2 at uhub2 port 1 configuration 1 interface 0 > uhidev2: vendor 0x04d9 product 0x1400, rev 1.10/1.43, addr 2, iclass 3/1 > ukbd1 at uhidev2: 8 modifier keys, 6 key codes > wskbd2 at ukbd1 mux 1 > wskbd2: connecting to wsdisplay0 > uhidev3 at uhub2 port 1 configuration 1 interface 1 > uhidev3: vendor 0x04d9 product 0x1400, rev 1.10/1.43, addr 2, iclass 3/1 > uhidev3: 3 report ids > ums1 at uhidev3 reportid 1: 5 buttons and Z dir. > wsmouse0 at ums1 mux 0 > uhid0 at uhidev3 reportid 2: input=3, output=0, feature=0 > uhid1 at uhidev3 reportid 3: input=1, output=0, feature=0 > dkcsum: sd0 matches BIOS drive 0x80 > root on sd0a swap on sd0b dump on sd0b > bnx1: address 00:19:b9:eb:75:d1 > brgphy0 at bnx1 phy 1: BCM5708C 10/100/1000baseT PHY, rev. 6 > bnx0: address 00:19:b9:eb:75:d3 > brgphy1 at bnx0 phy 1: BCM5708C 10/100/1000baseT PHY, rev. 6 > fmbraga:135$ > > Same pf.conf was applied: > > > > On Mon, Nov 12, 2007 at 02:05:54PM -0300, Fernando Braga wrote: > > > > > >Is there a way to minimize interrupts usage in this configuration ? > > > > > >My pf.conf is: > > > > > ># $OpenBSD: pf.conf,v 1.34 2007/02/24 19:30:59 millert Exp $ > > > > > >ext_if="bnx0" > > >int_if="bnx1" > > > > > >table <network-int> persist { 200.254.131.128/25 } > > >table <redes-biz> persist file "/etc/pf.biz" > > > > > >set skip on lo > > >set limit states 750000 > > > > > >scrub in > > > > > >altq on $int_if bandwidth 1000Mb hfsc queue { local, embratel } > > >altq on $ext_if bandwidth 1000Mb hfsc queue { local, embratel } > > > > > >queue local bandwidth 100Mb hfsc > > > > > >queue embratel bandwidth 200Mb hfsc(ecn realtime 200Mb upperlimit > > >200Mb ) \ > > > { Q-pri Q-icmp Q-vpn Q-biz Q-mail Q-http Q-ftp Q-def } > > > > > >queue Q-pri bandwidth 25Mb hfsc(realtime 15Mb linkshare 25Mb > > >upperlimit 180Mb) priority 7 > > >queue Q-icmp bandwidth 25Mb hfsc(realtime 15Mb linkshare 25Mb > > >upperlimit 30Mb) priority 7 > > >queue Q-vpn bandwidth 25Mb hfsc(realtime 15Mb linkshare 25Mb > > >upperlimit 180Mb) priority 6 > > >queue Q-biz bandwidth 25Mb hfsc(realtime 15Mb linkshare 25Mb > > >upperlimit 180Mb) priority 6 > > >queue Q-mail bandwidth 25Mb hfsc(realtime 15Mb linkshare 25Mb > > >upperlimit 180Mb) priority 4 > > >queue Q-http bandwidth 25Mb hfsc(realtime 15Mb linkshare 25Mb > > >upperlimit 180Mb) priority 4 > > >queue Q-ftp bandwidth 25Mb hfsc(realtime 15Mb linkshare 25Mb > > >upperlimit 180Mb) priority 4 > > >queue Q-def bandwidth 25Mb hfsc(default linkshare 25Mb > > >upperlimit 128Mb) priority 0 > > > > > >block in log > > >pass out > > > > > >antispoof quick for { lo $ext_if } > > >pass in quick on lo0 no state > > > > > >## Regras de merovingio -- bridge > > >pass in on $ext_if proto tcp to ($ext_if) port ssh queue(local) > > > > > >pass in on $int_if > > >queue(Q-def Q-pri) label "int-DEFA" > > >pass in on $int_if proto icmp > > >queue(Q-icmp ) label "int-ICMP" > > >pass in on $int_if proto gre > > >queue(Q-vpn ) label "int-VPN-gre" > > >pass in on $int_if proto esp > > >queue(Q-vpn ) label "int-VPN-esp" > > >pass in on $int_if proto ah > > >queue(Q-vpn ) label "int-VPN-ah" > > >pass in on $int_if proto l2tp > > >queue(Q-vpn ) label "int-VPN-l2tp" > > >pass in on $int_if proto { tcp udp } to port { 500 4500 } > > >queue(Q-vpn ) label "int-VPN-ipsec" > > >pass in on $int_if proto { tcp udp } to port { pptp } > > >queue(Q-vpn ) label "int-VPN-pptp" > > >pass in on $int_if proto tcp to port { snmp } > > >queue(Q-icmp Q-pri) label "int-ICMP-snmp" > > >pass in on $int_if proto tcp to port { www } > > >queue(Q-http Q-pri) label "int-HTTP" > > >pass in on $int_if proto tcp to port { https } > > >queue(Q-biz Q-pri) label "int-BIZZ-https" > > >pass in on $int_if proto { tcp udp } to port { domain ntp } queue( > > > Q-pri) label "int-PRII" > > > > > >pass in on $int_if to <redes-biz> > > >queue(Q-biz Q-pri) label "int-BIZZ-redes" > > >pass in on $int_if proto tcp to port { smtp pop3 imap 465 995 } > > >queue(Q-mail q-pri) label "int-MAIL" > > > > > >pass in on $ext_if > > >queue(Q-def Q-pri) label "ext-DEFA" > > >pass in on $ext_if proto icmp > > >queue(Q-icmp ) label "ext-ICMP" > > >pass in on $ext_if proto gre > > >queue(Q-vpn ) label "ext-VPN-gre" > > >pass in on $ext_if proto esp > > >queue(Q-vpn ) label "ext-VPN-esp" > > >pass in on $ext_if proto ah > > >queue(Q-vpn ) label "ext-VPN-ah" > > >pass in on $ext_if proto l2tp > > >queue(Q-vpn ) label "ext-VPN-l2tp" > > >pass in on $ext_if proto { tcp udp } to port { 500 4500 } > > >queue(Q-vpn ) label "ext-VPN-ipsec" > > >pass in on $ext_if proto { tcp udp } to port { pptp } > > >queue(Q-vpn ) label "ext-VPN-pptp" > > >pass in on $ext_if proto tcp to port { snmp } > > >queue(Q-icmp Q-pri) label "ext-ICMP-snmp" > > >pass in on $ext_if proto tcp to port { www } > > >queue(Q-http Q-pri) label "ext-HTTP" > > >pass in on $ext_if proto tcp to port { https } > > >queue(Q-biz Q-pri) label "ext-BIZZ-https" > > >pass in on $ext_if proto { tcp udp } to port { domain ntp } queue( > > > Q-pri) label "ext-PRII" > > > > > >pass in on $ext_if from <redes-biz> > > >queue(Q-biz Q-pri) label "ext-BIZZ-https" > > >pass in on $ext_if proto tcp to port { smtp pop3 imap 465 995 } > > >queue(Q-mail q-pri) label "ext-MAIL" > > > > > Kind regards, > > -- > Fernando M. Braga > +55 82 9985-4579 -- Those who can, do. Those who can't, sue.