Mark Mitchell wrote on :
> Chad Dougherty wrote:
>
>> The vulnerability note has been significantly reworked to focus on the
>> issue of undefined behavior handling in the compiler and the fact that
>> conforming implementations are not required to warn of this condition.
>> I've tried to incorporate many of the valid concerns that were raise on
>> this list in response to the original vulnerability note.
>
> Thank you for making the update; this is a big improvement.
>
> However, I'm surprised that only GCC is listed as "vulnerable" at the
> bottom of the page. We've provided information about a lot of other
> compilers that do the same optimization. Why is the status for
> compilers from Microsoft, Intel, IBM, etc. listed as "Unknown" instead
> of "Vulnerable"?
I would like to see CERT publish a consistent and reliable methodology for
their assessment of this vulnerability in various compilers, and the full
results obtained from running this test methodology against the various listed
compilers, in enough detail that anyone can reproduce them. Consistency,
commensurability and reproducibility are at the heart of the scientific
method.
rCs has been trying very hard to address the issues raised with some code
examples, but what we've ended up seeing is differing and incommensurable code
examples that have been evaluated by differing and incommensurable standards
of validation - sometimes evaluating the generated assembly, sometimes just
looking at the output of the program (which is an indirect inference at best),
even in one notable case just throwing up his hands because he found the
generated assembly "harder to examine". I don't want to be mean about it, but
someone who is not utterly fluent in inspecting compiler-generated assembly
code should probably not be tasked with performing this analysis. It does not
inspire any confidence in whatever other results you have obtained without
"showing us your working".
Some cases appear to only be demonstrable in an artificial testcase where
the integer is a known constant at compile time, and vary with the value of
that constant.
These results are then presented in a table which simply lists a single
status for each vendor, when most vendors ship multiple products which may
differ in their behaviour.
This is not just trivium or pedantry. The procedures you have followed are
seriously flawed and unscientific, patchy, ad-hoc and inconsistent. The data
is fudged. The conclusions cannot be relied upon.
The flaws in your methodology are so severe, and yet CERT's insistence on
trying to draw invalid conclusions from these tests is so virulent, that your
actions seem inexplicably biased. The way in which you are operating a
blatant double standard, where you present a behaviour in gcc as a serious
flaw where you excuse, overlook or ignore the exact same behaviour in every
other vendor's compiler, gives credence to what would otherwise be a ludicrous
conspiracy theory: that CERT is acting under politically-inspired direction to
attack Open Source software by spreading FUD about it being insecure. Not
being paranoid, I can see the alternative ("cock-up not conspiracy")
explanation: that this VU was written in a hurry based only on the
notification you received relating to gcc, and was not planned properly, nor
researched properly, nor written accurately, in the course of its rushed
creation. However, the longer and more insistently you attempt to defend the
indefensible process through which you have arrived at this invalid outcome,
the more you undermine that line of defence and make the
politically-motivated-attack-on-OSS theory plausible, and you are after all a
government agency, and we do after all have a documented history in recent
years of the administration interfering with the publications of its agencies
and demanding the suppression of scientifically accurate facts and their
replacement by politically inspired factually incorrect information. There is
no reason that we know of to suppose CERT would be any more or less immune to
this sort of pressure than, for example, federally-funded scientific bodies
studying climate change.
But, damaging though that is to CERT as an organisation, it's still beside
the point, because regardless of your motivation, the fact is that your
procedures have been unscientific and do not support the conclusions you have
attempted to draw from them.
This VU falls massively far below the standards we have come to expect from
CERT, and should be withdrawn and reworked from scratch, before it causes
further damage to your reputation for professionalism and independence.
cheers,
DaveK
--
Can't think of a witty .sigline today....
