> El vie, 12-10-2018 a las 17:46 +0000, mario.limoncie...@dell.com
> escribió:
> > > -----Original Message-----
> > > From: Dennis Gilmore <dgilm...@redhat.com>
> > > Sent: Friday, October 12, 2018 8:39 AM
> > > To: Pandruvada, Srinivas
> > > Cc: linux-kernel@vger.kernel.org; linux-a...@vger.kernel.org;
> > > Limonciello, Mario
> > > Subject: Re: issues with suspend on Dell XPS 13 2-in-1
> > >
> >
> > It appears I'm being added to this thread late. Can you give me some
> > more context?
> > Which XPS 2-in-1 is this (model number)? And what are the issues?
> >
> > I'm "guessing" high power consumption over S2I?
>
> Sorry we had a discussion back in April about high power consumption
> when suspending. The conversation stopped when the debugging steps did
> not work. Which turns out to be due to secure boot. I have just dealt
> with it by shutting down, but as I am currently travelling it annoyed
> me enough to try and figure out what is happening again. I have a 9635
> and esentially I get the same battery use regardless of if I suspend or
> not.
>
> Dennis
>
I see.
Can you remove those patches for now and perform those debugging steps?
>
> > >
> > > El jue, 26-04-2018 a las 15:09 +0000, Pandruvada, Srinivas
> > > escribió:
> > > > On Thu, 2018-04-26 at 07:42 -0500, Dennis Gilmore wrote:
> > > > > Hi Srinivas,
> > > > >
> > > > > El jue, 26-04-2018 a las 05:34 +0000, Pandruvada, Srinivas
> > > > > escribió:
> > > > > > Hi Dennis,
> > > > > >
> > > > > > On Wed, 2018-04-25 at 22:06 -0500, Dennis Gilmore wrote:
> > > > > > > Hi Srinivas,
> > > > > > >
> > > > > > > Yes I have latest bios, I have version 1.3.1 that was
> > > > > > > released
> > > > > > > on
> > > > > > > 18th
> > > > > > > of Feb.
> > > > > >
> > > > > > Can you try these commands and repeat the test?
> > > > > >
> > > > > > # cd /sys/kernel/debug/pmc_core/
> > > > > > # for i in {0..32}; do echo $i > ltr_ignore; done
> > > > >
> > > > > # for i in {0..32}; do echo $i > ltr_ignore; done
> > > > > -bash: ltr_ignore: Operación no permitida
> > >
> > > After some digging i figured out that the patches fedora has for
> > > secureboot caused the access to ltr_ignore to not work.
> > >
> >
> > If you are trying to debug high power consumption and those patches
> > are causing
> > problems, can you please remove those patches?
> >
> > LTR ignoring is an important debugging tactic to find problems with
> > S2I consuming
> > too much power.
> >
> > Additionally if you can read
> > /sys/kernel/debug/pmc_core/pch_ip_power_gating_status
> > That may help to point out what is wrong.
> >
> > > # turbostat
> > > turbostat version 18.07.27 - Len Brown <l...@kernel.org>
> > > CPUID(0): GenuineIntel 22 CPUID levels; family:model:stepping
> > > 0x6:8e:9
> > > (6:142:9)
> > > CPUID(1): SSE3 MONITOR SMX EIST TM2 TSC MSR ACPI-TM HT TM
> > > CPUID(6): APERF, TURBO, DTS, PTM, HWP, HWPnotify, HWPwindow,
> > > HWPepp,
> > > No-HWPpkg, EPB
> > > cpu1: MSR_IA32_MISC_ENABLE: 0x00850089 (TCC EIST MWAIT PREFETCH
> > > TURBO)
> > > CPUID(7): SGX
> > > cpu1: MSR_IA32_FEATURE_CONTROL: 0x0000ff07 (Locked )
> > > CPUID(0x15): eax_crystal: 2 ebx_tsc: 134 ecx_crystal_hz: 0
> > > TSC: 1608 MHz (24000000 Hz * 134 / 2 / 1000000)
> > > CPUID(0x16): base_mhz: 1600 max_mhz: 3600 bus_mhz: 100
> > > cpu1: MSR_MISC_PWR_MGMT: 0x00401cc0 (ENable-EIST_Coordination
> > > DISable-
> > > EPB DISable-OOB)
> > > RAPL: 58254 sec. Joule Counter Range, at 4 Watts
> > > cpu1: MSR_PLATFORM_INFO: 0x804043df1011000
> > > 4 * 100.0 = 400.0 MHz max efficiency frequency
> > > 16 * 100.0 = 1600.0 MHz base frequency
> > > cpu1: MSR_IA32_POWER_CTL: 0x0024005d (C1E auto-promotion: DISabled)
> > > cpu1: MSR_TURBO_RATIO_LIMIT: 0x22222224
> > > 34 * 100.0 = 3400.0 MHz max turbo 4 active cores
> > > 34 * 100.0 = 3400.0 MHz max turbo 3 active cores
> > > 34 * 100.0 = 3400.0 MHz max turbo 2 active cores
> > > 36 * 100.0 = 3600.0 MHz max turbo 1 active cores
> > > cpu1: MSR_CONFIG_TDP_NOMINAL: 0x0000000d (base_ratio=13)
> > > cpu1: MSR_CONFIG_TDP_LEVEL_1: 0x0006001c (PKG_MIN_PWR_LVL1=0
> > > PKG_MAX_PWR_LVL1=0 LVL1_RATIO=6 PKG_TDP_LVL1=28)
> > > cpu1: MSR_CONFIG_TDP_LEVEL_2: 0x00100038 (PKG_MIN_PWR_LVL2=0
> > > PKG_MAX_PWR_LVL2=0 LVL2_RATIO=16 PKG_TDP_LVL2=56)
> > > cpu1: MSR_CONFIG_TDP_CONTROL: 0x00000000 ( lock=0)
> > > cpu1: MSR_TURBO_ACTIVATION_RATIO: 0x0000000c
> > > (MAX_NON_TURBO_RATIO=12
> > > lock=0)
> > > cpu1: MSR_PKG_CST_CONFIG_CONTROL: 0x1e008008 (UNdemote-C3,
> > > UNdemote-
> > > C1,
> > > demote-C3, demote-C1, locked, pkg-cstate-limit=8 (unlimited))
> > > cpu1: POLL: CPUIDLE CORE POLL IDLE
> > > cpu1: C1: MWAIT 0x00
> > > cpu1: C1E: MWAIT 0x01
> > > cpu1: C3: MWAIT 0x10
> > > cpu1: C6: MWAIT 0x20
> > > cpu1: C7s: MWAIT 0x33
> > > cpu1: C8: MWAIT 0x40
> > > cpu1: C9: MWAIT 0x50
> > > cpu1: C10: MWAIT 0x60
> > > cpu1: cpufreq driver: intel_pstate
> > > cpu1: cpufreq governor: powersave
> > > cpufreq intel_pstate no_turbo: 0
> > > cpu1: MSR_MISC_FEATURE_CONTROL: 0x00000000 (L2-Prefetch L2-
> > > Prefetch-
> > > pair L1-Prefetch L1-IP-Prefetch)
> > > cpu0: MSR_PM_ENABLE: 0x00000001 (HWP)
> > > cpu0: MSR_HWP_CAPABILITIES: 0x01070d24 (high 36 guar 13 eff 7 low
> > > 1)
> > > cpu0: MSR_HWP_REQUEST: 0x80002404 (min 4 max 36 des 0 epp 0x80
> > > window
> > > 0x0 pkg 0x0)
> > > cpu0: MSR_HWP_INTERRUPT: 0x00000000 (Dis_Guaranteed_Perf_Change,
> > > Dis_Excursion_Min)
> > > cpu0: MSR_HWP_STATUS: 0x00000000 (No-Guaranteed_Perf_Change, No-
> > > Excursion_Min)
> > > cpu0: MSR_IA32_ENERGY_PERF_BIAS: 0x00000006 (balanced)
> > > cpu0: MSR_RAPL_POWER_UNIT: 0x000a0e03 (0.125000 Watts, 0.000061
> > > Joules,
> > > 0.000977 sec.)
> > > cpu0: MSR_PKG_POWER_INFO: 0x00000024 (4 W TDP, RAPL 0 - 0 W,
> > > 0.000000
> > > sec.)
> > > cpu0: MSR_PKG_POWER_LIMIT: 0x420078009c8048 (UNlocked)
> > > cpu0: PKG Limit #1: ENabled (9.000000 Watts, 24.000000 sec, clamp
> > > DISabled)
> > > cpu0: PKG Limit #2: DISabled (15.000000 Watts, 0.002441* sec, clamp
> > > DISabled)
> > > cpu0: MSR_DRAM_POWER_LIMIT: 0x5400de00000000 (UNlocked)
> > > cpu0: DRAM Limit: DISabled (0.000000 Watts, 0.000977 sec, clamp
> > > DISabled)
> > > cpu0: MSR_PP0_POLICY: 0
> > > cpu0: MSR_PP0_POWER_LIMIT: 0x00000000 (UNlocked)
> > > cpu0: Cores Limit: DISabled (0.000000 Watts, 0.000977 sec, clamp
> > > DISabled)
> > > cpu0: MSR_PP1_POLICY: 0
> > > cpu0: MSR_PP1_POWER_LIMIT: 0x00000000 (UNlocked)
> > > cpu0: GFX Limit: DISabled (0.000000 Watts, 0.000977 sec, clamp
> > > DISabled)
> > > cpu0: MSR_IA32_TEMPERATURE_TARGET: 0x0a640080 (100 C)
> > > cpu0: MSR_IA32_PACKAGE_THERM_STATUS: 0x882e0808 (54 C)
> > > cpu0: MSR_IA32_PACKAGE_THERM_INTERRUPT: 0x00000003 (100 C, 100 C)
> > > cpu1: MSR_PKGC3_IRTL: 0x0000884e (valid, 79872 ns)
> > > cpu1: MSR_PKGC6_IRTL: 0x00008876 (valid, 120832 ns)
> > > cpu1: MSR_PKGC7_IRTL: 0x00008894 (valid, 151552 ns)
> > > cpu1: MSR_PKGC8_IRTL: 0x000088fa (valid, 256000 ns)
> > > cpu1: MSR_PKGC9_IRTL: 0x0000894c (valid, 339968 ns)
> > > cpu1: MSR_PKGC10_IRTL: 0x00008bf2 (valid, 1034240 ns)
> > > Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz
> IRQ
> > > SMI POLL
> > > C1 C1E C3 C6 C7s C8 C9 C10 POLL%
> > > C1% C1E% C3% C6% C7s% C8% C9% C10% CPU
> > > %c1
> > > CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp
> > > GFX%rc6 GFXMHz Totl%C0 Any%C0
> > > GFX%C0 CPUGFX% Pkg%pc2 Pkg%pc3 Pkg%pc6
> > > Pkg%pc7 Pkg%pc8 Pkg%pc9 Pk%pc10
> > > CPU%LPI SYS%LPI PkgWatt CorWatt GFXWatt
> > > RAMWatt PKG_% RAM_%
> > > - - 238 14.13 1682 1607 7665 0 5
> > > 191 226 243 1561 0 1603 199 2418 0.00
> > > 0.24 0.27 0.12 4.510.00 7.03 3.45 70.
> > > 17 14.11
> > > 0.08 5.96 65.71 51 51 94.67 300 52.
> > > 99 44.18
> > > 5.13 3.41 12.05 40.26 0.00 0.00 0.00 0.0
> > > 00.00
> > > 0.00 0.00 3.37 1.73 0.02 0.47 0.00 0.0
> > > 0
> > > 0 0 215 14.00 1534 1607 2766 0 3
> > > 120 68 87 549 0 423 116 732 0.00
> > > 0.35 0.25 0.17 6.170.00 7.39 9.48 62.
> > > 09 10.17
> > > 0.10 7.33 68.40 49 51 94.67 300 53.
> > > 01 44.19
> > > 5.13 3.41 12.05 40.27 0.00 0.00 0.00 0.0
> > > 00.00
> > > 0.00 0.00 3.37 1.73 0.02 0.47 0.00 0.0
> > > 0
> > > 0 2 117 9.06 1297 1607 1600 0 1
> > > 16 76 73 394 0 512 34 693 0.00
> > > 0.08 0.47 0.14 4.500.00 8.65 1.55 75.
> > > 54 15.12
> > > 1 1 168 10.72 1569 1608 1459 0 1
> > > 30 48 40 342 0 306 42 581 0.00
> > > 0.32 0.28 0.08 3.680.00 5.47 2.63 76.
> > > 75 21.58
> > > 0.06 4.60 63.04 51
> > > 1 3 450 22.74 1979 1608 1840 0 0
> > > 25 34 43 276 0 362 7 412 0.00
> > > 0.21 0.07 0.09 3.680.00 6.60 0.14 66.
> > > 29 9.57
> > > Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz
> IRQ
> > > SMI POLL
> > > C1 C1E C3 C6 C7s C8 C9 C10 POLL%
> > > C1% C1E% C3% C6% C7s% C8% C9% C10% CPU
> > > %c1
> > > CPU%c3 CPU%c6 CPU%c7 CoreTmp PkgTmp
> > > GFX%rc6 GFXMHz Totl%C0 Any%C0
> > > GFX%C0 CPUGFX% Pkg%pc2 Pkg%pc3 Pkg%pc6
> > > Pkg%pc7 Pkg%pc8 Pkg%pc9 Pk%pc10
> > > CPU%LPI SYS%LPI PkgWatt CorWatt GFXWatt
> > > RAMWatt PKG_% RAM_%
> > > - - 292 19.04 1535 1608 10249 12 18
> > > 455 702 509 2538 1 1737 190 2748 0.00
> > > 0.37 0.47 0.31 6.330.00 7.04 2.37 64.
> > > 05 14.51
> > > 0.19 6.56 59.70 49 50 90.05 300 63.
> > > 43 45.18
> > > 9.83 4.95 11.88 36.32 0.00 0.00 0.00 0.0
> > > 00.00
> > > 0.00 0.00 3.33 1.63 0.04 0.56 0.00 0.0
> > > 0
> > > 0 0 304 20.87 1459 1608 3771 3 7
> > > 106 156 119 824 0 527 79 714 0.01
> > > 0.32 0.48 0.29 9.520.00 8.61 5.20 54.
> > > 69 18.23
> > > 0.16 6.46 54.29 49 50 90.05 300 63.
> > > 43 45.18
> > > 9.83 4.95 11.88 36.32 0.00 0.00 0.00 0.0
> > > 00.00
> > > 0.00 0.00 3.33 1.63 0.04 0.56 0.00 0.0
> > > 0
> > > 0 2 464 23.30 1992 1608 2368 3 3
> > > 79 188 98 426 0 436 20 588 0.00
> > > 0.28 0.30 0.29 3.870.00 6.92 0.88 64.
> > > 04 15.79
> > > 1 1 175 13.61 1283 1608 2103 3 6
> > > 169 154 147 644 0 455 67 760 0.00
> > > 0.49 0.64 0.31 6.030.00 7.60 2.95 68.
> > > 42 14.40
> > > 0.22 6.66 65.11 49
> > > 1 3 226 18.39 1227 1608 2007 3 2
> > > 101 204 145 644 1 319 24 686 0.00
> > > 0.38 0.47 0.35 5.920.01 5.02 0.45 69.
> > > 04 9.62
> > > Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz
> IRQ
> > > SMI
> > > POLL C1 C1E C3 C
> > > 6 C7s C8 C9 C10
> > > POLL% C1% C1E% C3% C6% C7s% C8% C9% C10
> > > %
> > > CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp
> > > PkgTmp GFX%rc6 GFXMHz Totl%C0
> > > Any%C0 GFX%C0 CPUGFX% Pkg%pc2 Pkg%pc3
> > > Pkg%pc6 Pkg%pc7 Pkg%pc8
> > > Pkg%pc9 Pk%pc10 CPU%LPI SYS%LPI PkgWatt
> > > CorWatt GFXWatt RAMWatt
> > > PKG_% RAM_%
> > > - - 38 1.97 1913 1623 -191291 16
> > > 320 5687 2001 711 324
> > > 8 2 2630 232 2606
> > > 0.00 0.08 0.06 0.03 0.46 0.00 0.65 0.1
> > > 2 5.58
> > > 1.69 0.02 0.35 95.98 56 56 100.70 300
> > > 6.45
> > > 3.88 0.20 0.16 0.32 5.20 0.00 0.00 0.0
> > > 3 0.00
> > > 89.94 90.31 0.00 0.91 0.22 0.00 0.28 0.0
> > > 0 0.00
> > > 0 0 42 2.18 1929 1623 -51859 4
> > > 93 1696 611 179 74
> > > 6 0 721 86 792
> > > 0.00 0.09 0.07 0.03 0.39 0.00 0.70 0.1
> > > 4 5.46
> > > 1.58 0.02 0.32 95.91 56 56 100.70 300
> > > 6.45
> > > 3.88 0.20 0.16 0.32 5.20 0.00 0.00 0.0
> > > 3 0.00
> > > 89.94 90.31 0.00 0.91 0.22 0.00 0.28 0.0
> > > 0 0.00
> > > 0 2 39 1.95 1974 1623 -44474 4
> > > 134 2280 672 214 89
> > > 3 2 902 40 430
> > > 0.00 0.13 0.08 0.04 0.52 0.00 0.90 0.1
> > > 4 5.31
> > > 1.81
> > > 1 1 42 2.20 1891 1623 -45616 4
> > > 6 369 234 105 63
> > > 0 0 310 33 622
> > > 0.00 0.02 0.04 0.02 0.41 0.00 0.31 0.0
> > > 4 6.02
> > > 1.36 0.01 0.37 96.06 55
> > > 1 3 29 1.56 1841 1623 -49342 4
> > > 87 1342 484 213 97
> > > 9 0 697 73 762
> > > 0.00 0.08 0.06 0.04 0.53 0.00 0.68 0.1
> > > 7 5.52
> > > 2.00
> > > Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz
> IRQ
> > > SMI
> > > POLL C1 C1E C3 C
> > > 6 C7s C8 C9 C10
> > > POLL% C1% C1E% C3% C6% C7s% C8% C9% C10
> > > %
> > > CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp
> > > PkgTmp GFX%rc6 GFXMHz Totl%C0
> > > Any%C0 GFX%C0 CPUGFX% Pkg%pc2 Pkg%pc3
> > > Pkg%pc6 Pkg%pc7 Pkg%pc8
> > > Pkg%pc9 Pk%pc10 CPU%LPI SYS%LPI PkgWatt
> > > CorWatt GFXWatt RAMWatt
> > > PKG_% RAM_%
> > > - - 810 43.00 1883 1608 25029 0
> > > 57 3373 1585 864 512
> > > 1 1 4401 559 2347
> > > 0.01 1.25 1.20 0.62 13.02 0.00 17.74 4.1
> > > 4 18.47
> > > 30.05 0.49 10.40 16.06 51 51 95.64 300
> > > 137.62
> > > 98.15 4.11 4.02 0.00 0.00 0.00 0.00 0.0
> > > 0 0.00
> > > 0.00 0.00 0.00 6.44 4.66 0.02 0.66 0.0
> > > 0 0.00
> > > 0 0 719 40.39 1779 1608 6804 0
> > > 12 817 383 225 149
> > > 5 0 1207 196 545
> > > 0.01 1.19 1.12 0.66 16.23 0.00 19.77 6.3
> > > 3 13.80
> > > 23.34 0.67 13.51 22.09 51 51 95.64 300
> > > 137.62
> > > 98.16 4.11 4.02 0.00 0.00 0.00 0.00 0.0
> > > 0 0.00
> > > 0.00 0.00 0.00 6.44 4.66 0.02 0.66 0.0
> > > 0 0.00
> > > 0 2 670 34.80 1924 1608 6507 0
> > > 11 683 442 253 149
> > > 6 0 1458 147 697
> > > 0.01 1.27 1.54 0.66 13.80 0.00 22.91 4.3
> > > 7 20.27
> > > 28.93
> > > 1 1 1013 52.58 1927 1608 5901 0
> > > 11 604 263 176 94
> > > 5 1 799 132 577
> > > 0.01 0.97 0.75 0.53 9.60 0.00 13.11 4.0
> > > 6 17.59
> > > 29.79 0.31 7.30 10.02 50
> > > 1 3 838 44.23 1894 1608 5817 0
> > > 23 1269 497 210 118
> > > 5 0 937 84 528
> > > 0.02 1.56 1.38 0.61 12.45 0.00 15.15 1.7
> > > 9 22.21
> > > 38.14
> > > Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz
> IRQ
> > > SMI
> > > POLL C1 C1E C3 C
> > > 6 C7s C8 C9 C10
> > > POLL% C1% C1E% C3% C6% C7s% C8% C9% C10
> > > %
> > > CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp
> > > PkgTmp GFX%rc6 GFXMHz Totl%C0
> > > Any%C0 GFX%C0 CPUGFX% Pkg%pc2 Pkg%pc3
> > > Pkg%pc6 Pkg%pc7 Pkg%pc8
> > > Pkg%pc9 Pk%pc10 CPU%LPI SYS%LPI PkgWatt
> > > CorWatt GFXWatt RAMWatt
> > > PKG_% RAM_%
> > > - - 801 44.72 1893 1522 28623 0
> > > 168 8252 2500 1022 437
> > > 3 0 2917 544 3699
> > > 0.04 2.25 1.60 0.70 9.11 0.00 10.08 4.2
> > > 2 23.80
> > > 33.40 1.29 6.49 14.10 54 54 89.74 300
> > > 144.45
> > > 95.89 5.02 4.67 0.00 0.00 0.00 0.00 0.0
> > > 0 0.00
> > > 0.00 0.00 0.00 6.29 4.59 0.03 0.65 0.0
> > > 0 0.00
> > > 0 0 733 40.67 1904 1522 6879 0
> > > 57 1930 691 307 133
> > > 3 0 752 212 948
> > > 0.05 2.20 1.64 0.87 11.01 0.00 10.18 7.8
> > > 1 21.97
> > > 41.31 0.55 6.84 10.63 52 54 89.74 300
> > > 144.45
> > > 95.89 5.02 4.67 0.00 0.00 0.00 0.00 0.0
> > > 0 0.00
> > > 0.00 0.00 0.00 6.29 4.59 0.03 0.65 0.0
> > > 0 0.00
> > > 0 2 937 50.69 1952 1522 6332 0
> > > 38 1968 641 226 96
> > > 4 0 736 128 710
> > > 0.04 2.25 1.46 0.66 7.81 0.00 10.51 3.8
> > > 6 19.43
> > > 31.29
> > > 1 1 851 48.25 1864 1522 5826 0
> > > 35 1648 589 194 87
> > > 2 0 482 118 969
> > > 0.03 2.02 1.62 0.45 7.01 0.00 6.57 3.7
> > > 8 26.97
> > > 26.01 2.03 6.13 17.58 54
> > > 1 3 685 39.28 1843 1522 9586 0
> > > 38 2706 579 295 120
> > > 4 0 947 86 1072
> > > 0.04 2.55 1.68 0.82 10.59 0.00 13.05 1.4
> > > 1 26.84
> > > 34.98
> > > ^CCore CPU Avg_MHz Busy% Bzy_MHz TSC_MHz
> IRQ
> > > SMI
> > > POLL C1 C1E C3 C
> > > 6 C7s C8 C9 C10
> > > POLL% C1% C1E% C3% C6% C7s% C8% C9% C10
> > > %
> > > CPU%c1 CPU%c3 CPU%c6 CPU%c7 CoreTmp
> > > PkgTmp GFX%rc6 GFXMHz Totl%C0
> > > Any%C0 GFX%C0 CPUGFX% Pkg%pc2 Pkg%pc3
> > > Pkg%pc6 Pkg%pc7 Pkg%pc8
> > > Pkg%pc9 Pk%pc10 CPU%LPI SYS%LPI PkgWatt
> > > CorWatt GFXWatt RAMWatt
> > > PKG_% RAM_%
> > > - - 1318 58.01 2462 1484 5393 0
> > > 37 2229 593 199 94
> > > 4 0 422 61 261
> > > 0.05 4.19 2.80 1.23 14.53 0.00 9.50 1.5
> > > 0 4.15
> > > 36.81 0.24 2.94 2.00 58 58 33.73 300
> > > 178.58
> > > 99.15 63.65 62.57 0.00 0.00 0.00 0.00 0.0
> > > 0 0.00
> > > 0.00 0.00 0.00 10.04 8.11 0.18 0.83 0.0
> > > 0 0.00
> > > 0 0 1943 78.79 2674 1483 1233 0
> > > 3 192 28 22 10
> > > 8 0 55 6 20
> > > 0.01 2.22 0.36 0.38 9.15 0.00 5.46 0.1
> > > 8 0.77
> > > 21.19 0.00 0.01 0.00 57 58 33.73 300
> > > 178.66
> > > 99.20 63.68 62.59 0.00 0.00 0.00 0.00 0.0
> > > 0 0.00
> > > 0.00 0.00 0.00 10.04 8.11 0.18 0.83 0.0
> > > 0 0.00
> > > 0 2 1139 52.89 2334 1484 1436 0
> > > 8 546 137 54 28
> > > 0 0 176 3 54
> > > 0.06 3.37 3.12 1.49 14.02 0.00 15.09 0.3
> > > 7 5.08
> > > 47.10
> > > 1 1 1113 49.66 2428 1484 1553 0
> > > 13 853 208 65 28
> > > 5 0 117 19 111
> > > 0.06 6.25 3.63 1.72 16.95 0.00 10.30 1.4
> > > 4 5.64
> > > 39.99 0.47 5.87 4.01 58
> > > 1 3 1076 50.71 2299 1485 1171 0
> > > 13 638 220 58 27
> > > 1 0 74 33 76
> > > 0.05 4.89 4.10 1.33 18.02 0.00 7.17 4.0
> > > 1 5.09
> > > 38.95
