Derek M Jones <[EMAIL PROTECTED]> wrote:
Figure 2 is evidence that there is a strong correlation between
developer performance and certain kinds of operator occurrences
in source code and no correlation for other kinds of measurements
of operators occurrences.
Er, no.
Figure 2 (left), with r = 0.64, is a WEAK correlation.
Remember, to grasp the importance of a correlation, you have to square it.
R-squared for (% correct answers) against (% occurrence in operator pairs
in specified sample of C code) is just 41%.
However, that's because the linear relationship measured by Pearson
correlation does *not* match what's actually happening in the data of
figure 2, which is quadratic relationship with maximum about 10%.
And that says "up to a certain point, the commoner the operator the
better the performance, beyond that point, the commoner the operator
the WORSE the performance."
Figure 2 (right) shows, well, basically all Figure 2 (right) really
shows is that ==, !=, +, and - are common and all the other operators
are rare. But you can get a nice quadratic relationship out of that too.
But there's an easily overlooked point here:
the horizontal axes of these figures are based on operator occurrences
in a certain body of C code. We have NO evidence that any of the
subjects have been exposed to ANY of that body of code, and the paper
gives NO evidence that that body of code is similar to the code that
the subjects (or other C developers in general) have been exposed to.
Let's give an example of that. I just counted the operators in a program
I happen to use a lot. The program has 250 000 lines in .c files; I did
not count .h files. Of those 250 000 lines, 110 000 lines are SLOC; they
contain a total of 133 000 operators (I counted assignment operators and
also counted ?: as an operator). 37 000 were the simple assignment
operator. Counting only the operators listed in the paper's table 2,
the program has 91 000 operators, about 10% of the code volume used in
the paper.
In table 2,
+ was twice as common as -
* was twice as common as /
& was commoner than *
&& was commoner than ||
== was four times as common as >
In the program I checked,
- was four times as common as +
* was ten times as common as /
* was four times as common as &
|| was a tiny bit commoner than &&
> was three times as common as ==
The only conclusion I want to draw from this is that the horizontal
axes in Jones's Figure 2 CANNOT be taken as surrogates for the amount
of exposure any developer has had to C operators; the relative frequencies
in different programs can be drastically different. That is, the
horizontal axes are about one body of code, while the vertical axes are
about developers exposed to a *different* body of code, quite likely
with different characteristics.
(Looking at table 2 one becomes a little suspicious: high frequency
operators usually occur often in pairs. But not '&'. Were some of
the occurrences of '&' really occurrences of the unary address-of operator?)
----------------------------------------------------------------------
PPIG Discuss List (discuss@ppig.org)
Discuss admin: http://limitlessmail.net/mailman/listinfo/discuss
Announce admin: http://limitlessmail.net/mailman/listinfo/announce
PPIG Discuss archive: http://www.mail-archive.com/discuss%40ppig.org/
