You present some valid points there.
I think we could first obtain a value via get_nprocs() and then try to
do some intelligent parsing on our own. If we get a bad value we can
just fallback to what we got from get_nprocs().
The main issue here is indeed differences between architectures. Also,
lets not forget our platform may not be Linux and /proc may not exist at
all.
As for HT being present, if it's there htt (or was it ht?) would appear
in "flags".
Cheers,
Alex
Preston Smith wrote:
> I seem to recall from previous times that this came up, that being
> able to reliably
> parse /proc/cpuinfo was an issue.. get_nprocs() is a glibc call that's
> *always* going
> to be there, on all linux kernels and architectures. It's a safe
> alternative, as it reports what
> Linux thinks is the number of processors, even though it's not smart
> about threads
> and physical chips.
>
> Perhaps using this patch's functionality as a selectable option for
> users of 2.6.x
> kernel, x86 linux, but leaving get_nprocs() as a safe default is a
> wise way to go?
>
> Things like this aren't reliably consistent between kernel versions,
> vendor
> kernels vs kernel.org kernels, (redhat kernels might have some lines, but
> vanilla doesn't..) and architectures. (does this dual-core/ht logic
> work on a dual-core G5?)
>
> And figuring out if ht is present? what about if ht is *possible* on
> that processor but
> disabled?
>
> -Preston
>
> some examples:
> core id: present in kernel.org 2.6.14
> not in kernel.org 2.4.31 or 2.4.21-37.EL (rhel3)
>
> physical id: not in kernel.org 2.4.31, but is in 2.4.21-37.EL
>
> 2.6.8 (powerpc) or 2.6.12.6 (sparc) has neither "core id" or "physical
> id".
>
>
>
> some /proc/cpuinfos follow:
> ---------------------------------------------------------------
>
> PowerPC:
> processor : 0
> cpu : 7400, altivec supported
> temperature : 23-25 C (uncalibrated)
> clock : 500MHz
> revision : 2.9 (pvr 000c 0209)
> bogomips : 995.32
>
> processor : 1
> cpu : 7400, altivec supported
> temperature : 47-49 C (uncalibrated)
> clock : 500MHz
> revision : 2.9 (pvr 000c 0209)
> bogomips : 995.32
>
> total bogomips : 1990.65
> machine : PowerMac3,3
> motherboard : PowerMac3,3 MacRISC Power Macintosh
> detected as : 65 (PowerMac G4 AGP Graphics)
> pmac flags : 00000004
> L2 cache : 1024K unified
> memory : 320MB
> pmac-generation : NewWorld
>
>
>
> Sparc:
>
> cpu : TI UltraSparc II (BlackBird)
> fpu : UltraSparc II integrated FPU
> promlib : Version 3 Revision 27
> prom : 3.27.0
> type : sun4u
> ncpus probed : 1
> ncpus active : 1
> Cpu0Bogo : 587.77
> Cpu0ClkTck : 0000000011a45229
> MMU Type : Spitfire
> State:
> CPU0: online
>
>
>
> On Mar 2, 2006, at 5:21 AM, Alex Balk wrote:
>
>> This doesn't seem to apply to all SMP systems (or is it kernel
>> version?):
>>
>>
>> kernel: 2.6.12-10-686-smp
>>
>>
>>
>> processor : 0
>> vendor_id : GenuineIntel
>> cpu family : 6
>> model : 8
>> model name : Pentium III (Coppermine)
>> stepping : 1
>> cpu MHz : 731.523
>> cache size : 256 KB
>> fdiv_bug : no
>> hlt_bug : no
>> f00f_bug : no
>> coma_bug : no
>> fpu : yes
>> fpu_exception : yes
>> cpuid level : 2
>> wp : yes
>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>> mca cmov pat pse36 mmx fxsr sse
>> bogomips : 1449.98
>>
>> processor : 1
>> vendor_id : GenuineIntel
>> cpu family : 6
>> model : 8
>> model name : Pentium III (Coppermine)
>> stepping : 1
>> cpu MHz : 731.523
>> cache size : 256 KB
>> fdiv_bug : no
>> hlt_bug : no
>> f00f_bug : no
>> coma_bug : no
>> fpu : yes
>> fpu_exception : yes
>> cpuid level : 2
>> wp : yes
>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>> mca cmov pat pse36 mmx fxsr sse
>> bogomips : 1462.27
>>
>>
>>
>> As for the code, perhaps these changes to your patch would provide with
>> the required results:
>>
>>
>> #define UNIPROCESSOR 1
>>
>>
>> /* dont_count_ht_cpus indicates an option has been set in the config */
>>
>> if ((n = sysconf(_SC_NPROCESSORS_ONLN) > UNIPROCESSOR &&
>> dont_count_ht_cpus) {
>>
>> /* examine /proc/cpuinfo */
>>
>> cores_in_cpuinfo = ....
>>
>> }
>>
>>
>> /* if cores_in_cpuinfo reports 0, we're on a system which doesn't report
>> core_id */
>>
>> if (cores_in_cpuinfo)
>>
>> return (int) cores_in_cpuinfo;
>>
>> else
>>
>> return (int) n;
>>
>>
>>
>> Also, MAX_CPUS is a hard-coded scalability limit. To avoid that maybe
>> you could just grab the last "processor" entry and assign an array of
>> that size. This would also cover hot-plugging CPUs, but might involve a
>> race in such a setup. What do you guys think?
>>
>> Cheers,
>> Alex
>>
>>
>> James Mcininch wrote:
>>
>>>
>>> On SMP machines, each entry in /proc/cpuinfo reports 'core id'
>>> and 'physical id' in addition to 'processor'. The number of
>>> 'processor' entries is cpu_count_virtual. The number of distinct
>>> 'physical id' entries is the number of CPU packages installed
>>> on the motherboard. The number of distinct 'core id' entries is
>>> the number of CPU cores.
>>>
>>> On non-SMP machines, each entry in /proc/cpuinfo reports
>>> 'cpu count' (if applicable) to denote how many 'processor'
>>> entries exist for the one physical CPU (i.e., if 'cpu count' is
>>> 2, that processor entry is 1/2 of the entries reported for that
>>> 1 physical CPU). So the sum of 'processor' entries divided by
>>> their respective 'cpu count' values is the cpu_count_cores and
>>> cpu_count_physical, and the number of 'processor' entries is the
>>> cpu_count_virtual.
>>>
>>> It's a little daft, but that's the way it seems to work.
>>>
>>> My original patch submission didn't handle the non-SMP case --
>>> I need to fix that.
>>>
>>>
>>> <[EMAIL PROTECTED]> wrote on 03/01/2006 12:26:47 PM:
>>>
>>>> From the look of /proc/stat and /proc/cpuinfo in cygwin, I am
>>>> not sure how you resolve between virtual and real processors.
>>>> Am I missing something?
>>>>
>>>> regards,
>>>> richard
>>>>
>>>> cpu 91283593 0 24818640 2641219155
>>>> cpu0 33563843 0 13294859 1331802093
>>>> cpu1 57719750 0 11523781 1309417062
>>>> page 28101525 3278915
>>>> swap 28101525 3042074
>>>> intr 1021797193
>>>> ctxt 2106977704
>>>> btime 1139854968
>>>>
>>>>
>>>> processor : 0
>>>> vendor_id : GenuineIntel
>>>> type : primary processor
>>>> cpu family : 15
>>>> model : 2
>>>> model name : Intel(R) Pentium(R) 4 CPU 2.80GHz
>>>> stepping : 9
>>>> brand id : 9
>>>> cpu count : 2
>>>> apic id : 0
>>>> cpu MHz : 2793
>>>> fpu : yes
>>>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>>>> mca cmov pat pse36 clfl dtes acpi mmx fxsr sse
>>>> e2 ss htt tmi pbe cid
>>>> processor : 1
>>>> vendor_id : GenuineIntel
>>>> type : primary processor
>>>> cpu family : 15
>>>> model : 2
>>>> model name : Intel(R) Pentium(R) 4 CPU 2.80GHz
>>>> stepping : 9
>>>> brand id : 9
>>>> cpu count : 2
>>>> apic id : 1
>>>> cpu MHz : 2793
>>>> fpu : yes
>>>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>>>> mca cmov pat pse36 clfl dtes acpi mmx fxsr sse
>>>> e2 ss htt tmi pbe cid
>>>>
>>>> -----Original Message-----
>>>> From: [EMAIL PROTECTED]
>>>> [mailto:[EMAIL PROTECTED] On Behalf Of
>>>> Jason A. Smith
>>>> Sent: 01 March 2006 15:42
>>>> To: James Mcininch
>>>> Cc: Ganglia Developers
>>>> Subject: Re: [Ganglia-developers] hyperthreading & cpu_num patch.
>>>>
>>>>
>>>> Hi James,
>>>>
>>>> As you mentioned in your bug report:
>>>>
>>>> http://bugzilla.ganglia.info/cgi-bin/bugzilla/show_bug.cgi?id=84
>>>>
>>>> This has been discussed several times in the mailing lists, like this
>>>> one last year:
>>>>
>>>> http://sourceforge.net/mailarchive/forum.php?forum_id=7186&max_rows=25&s
>>>>
>>>> tyle=threaded&viewmonth=200502
>>>>
>>>> But as the discussion last year confirms, different people might
>>>> want it
>>>> to work in different ways. The best solution would probably be to
>>>> patch
>>>> gmond to report both the number of real and virtual (hyperthreaded)
>>>> CPUs
>>>> and have a config option in the webfrontend to display the load
>>>> based on
>>>> one of the two CPU counts.
>>>>
>>>> ~Jason
>>>>
>>>>
>>>> On Wed, 2006-03-01 at 09:40 -0500, James Mcininch wrote:
>>>>>
>>>>> I want to bring attention to Bugzilla item 84 -- a patch submitted to
>>>>> fix improper reporting of cpu_num on hyperthreaded Intel processors.
>>>>>
>>>>> The patch causes it to report the number of CPU cores rather than the
>>>>> number of virtual CPUs.
>>>>>
>>>> --
>>>> /------------------------------------------------------------------\
>>>> | Jason A. Smith Email: [EMAIL PROTECTED] |
>>>> | Atlas Computing Facility, Bldg. 510M Phone: (631)344-4226 |
>>>> | Brookhaven National Lab, P.O. Box 5000 Fax: (631)344-7616 |
>>>> | Upton, NY 11973-5000 |
>>>> \------------------------------------------------------------------/
>>>>
>>>>
>>>>
>>>>
>>>> -------------------------------------------------------
>>>> This SF.Net email is sponsored by xPML, a groundbreaking scripting
>>>> language that extends applications into web and mobile media.
>>>> Attend the
>>>> live webcast and join the prime developer group breaking into this new
>>>> coding territory!
>>>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
>>>>
>>>> _______________________________________________
>>>> Ganglia-developers mailing list
>>>> Ganglia-developers@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/ganglia-developers
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> For more information about Barclays Capital, please
>>>> visit our web site at http://www.barcap.com.
>>>>
>>>>
>>>> Internet communications are not secure and therefore the Barclays
>>>> Group does not accept legal responsibility for the contents of this
>>>> message. Although the Barclays Group operates anti-virus programmes,
>>>> it does not accept responsibility for any damage whatsoever that is
>>>> caused by viruses being passed. Any views or opinions presented are
>>>> solely those of the author and do not necessarily represent those of
>>> the
>>>> Barclays Group. Replies to this email may be monitored by the
>>>> Barclays
>>>> Group for operational or business reasons.
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>
>>
>> -------------------------------------------------------
>> This SF.Net email is sponsored by xPML, a groundbreaking scripting
>> language
>> that extends applications into web and mobile media. Attend the live
>> webcast
>> and join the prime developer group breaking into this new coding
>> territory!
>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
>> _______________________________________________
>> Ganglia-developers mailing list
>> Ganglia-developers@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/ganglia-developers
>
> --
> Preston Smith <[EMAIL PROTECTED]>
> Systems Research Engineer
> Rosen Center for Advanced Computing, Purdue University
>
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting
> language
> that extends applications into web and mobile media. Attend the live
> webcast
> and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Ganglia-developers mailing list
> Ganglia-developers@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/ganglia-developers
