On Thu, Dec 15, 2011 at 04:55:16AM -0600, Michael Larabel wrote:
> On 12/15/2011 04:41 AM, Michael Ross wrote:
> >Am 15.12.2011, 11:10 Uhr, schrieb Michael Larabel
> ><michael.lara...@phoronix.com>:
> >
> >>On 12/15/2011 02:48 AM, Michael Ross wrote:
> >
> >>>Anyway these tests were performed on different hardware, FWIW.
> >>>And with different filesystems, different compilers, different GUIs...
> >>>
> >>>
> >>
> >>No, the same hardware was used for each OS.
> >>
> >
> >The picture under the heading "System Hardware / Software" does
> >not reflect that.
> >
> >Motherboard description differs, Chipset description for FreeBSD
> >is empty.
> >
>
> I was the on that carried out the testing and know that it was on
> the same system.
>
> All of the testing, including the system tables, is fully automated.
> Under FreeBSD sometimes the parsing of some component strings isn't
> as nice as Linux and other supported operating systems by the
> Phoronix Test Suite. For the BSD motherboard string parsing it's
> grabbing hw.vendor/hw.product from sysctl.
>
> Is there a better place to read the motherboard DMI information from?
I *think* what you're referring to is SMBIOS strings -- and these are
available from kenv(1) / kenv(2), not sysctl. But keep reading for why
SMBIOS data is not 100% reliable (greatly depends on the hardware). For
actual device strings/etc. for all devices on busses (PCI, AGP, etc.)
you can use pciconf -lvcb.
That's about as good as it's going to get via software. SMBIOS data
(e.g. smbios.{bios,chassis,planar,system}) is never going to give you
fully-identifiable data; I can point you to tons of systems where the
data inserted there is nonsense, sometimes even just ASCII spaces (and
that is the fault of the system vendor/BIOS manufacturer, not FreeBSD).
Sometimes identical strings are used across completely different
systems/boards (sometimes even server-class boards like ones from
Supermicro). And PCI vendor strings don't give you things like speeds,
frequency/voltages, etc.. Sometimes this matters. For example (just
making something up): "the video benchmark was horrible on FreeBSD",
when in fact it turned out that a run of "pciconf -lvcb" showed your
PCIe card was running at x4 link speed instead of x16.
The best place to get your specifications from are:
* The box
* The physical hardware (by physically inspecting it)
* The user manual / product documentation/
* Purchase orders from whoever bought the hardware
* And, of course, operational speed (if possible) from the OS/userland
utilities
When I read a benchmark/review, I have to assume the person is doing
them on a system they have 100% control over, all the way down to the
hardware. Thus, they should know what exact hardware they have.
Also, when publishing results online, you should take the time to
proofread everything (with a 2nd set of eyes if possible) and be patient
and thorough. People like accuracy, especially when there's hard
data/evidence to back it up that can be made available for download.
Try to understand: so many review-esque sites consist of individuals who
do not understand even remotely what they're doing.
I'm going to give you two examples -- one personal, one word-of-mouth
but from someone I trust dearly.
I have a "reverse analysis" of Anantech's Intel 510 SSD review that has been
sitting in my "draft" folder on my blog for a month now because I'm
downright afraid to publish how their data seems completely and totally
wrong (with evidence to prove it). I'm afraid/stalling because I want
to make absolutely damn sure I'm not missing some key piece of evidence
that explains it, and I've had multiple people read it and go "...wow, I
didn't notice that, that benchmark data makes no sense", but I'm STILL
reluctant. The last thing I want to do is "publish" something that
sparks a controversy where it turns out I'm wrong (and I AM wrong, quite
often!).
As for the other:
http://www.overclockers.com/bulldozer-architecture-explained/
The author of this "review" talks about CPU arch and is praised for
writing a "wonderful article that speaks the truth". But sadly that
doesn't appear to be the case. A colleague of mine is long-time friends
with another individual who is getting his Ph.D in computer architecture
and recently submit a paper to a journal (and was published/accepted)
which has published papers on things like RAID (when it was first
introduced as a concept/method), and hardware watchpoints. Said
individual read the above "review" and described it as, quote, "the
worst article on computer architecture on the entire Internet". One of
the amusing quotes (that got me laughing since I did understand it; my
understanding of CPUs on a silicon level is limited, I'm just an old
65xxx assembly programmer...) was how the article states "this is the
first time AMD has implemented branch prediction". Sigh.
Here's the kicker: said individual immediately recognised that the
article was a near dry cut-and-paste from one of two commonly-used
computer architecture books in college/universities; the first book is
basically a "beginner's guide to CPU architecture". The book is also a
bit old at that. Individual proceeded to look up where the article
author went to school, and noted that said school's CPU architecture
course **ends** with that book.
The user/viewer demographic of overclockers.com is going to be
significantly different from that of phoronix.com -- you know that I'm
sure. The point is that you should be aware that there is going to
be significant discussions that come from publishing such benchmark
comparisons with such a demographic. Things that indicate severe
performance differential (e.g. "10x to 100x worse") are going to be
focused on and criticised -- and hopefully in a socially-agreeable
manner[1] -- and in a much different way than, say, a 3D video card
review site ("lol ur pc sux if u spend onl $4000 on it lol").
The first step is to try and figure out what exactly you're seeing and
why it's so significantly different when compared to other OSes.
[1]: I'm sure by now you know that the BSDs in general tend to harbour a
community of folks who are more argumentative/aggressive than, say,
Linux (generally speaking). In this thread though, I think all of us
really want to assist in some way to figure out what exactly is going on
here, scheduler-wise, and see if we can put something together to hand
developers who are "responsible" for said code and see what comes of it.
Remember, we're all here to try and make things better... I hope. :-)
Footnote: It's nice meeting you (indirectly), I was always curious who
did the phoronix.com reviews/"stuff" when it came to FreeBSD.
Greetings!
--
| Jeremy Chadwick jdc at parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, US |
| Making life hard for others since 1977. PGP 4BD6C0CB |
_______________________________________________
freebsd-performance@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscr...@freebsd.org"
