Risks of Risk Assessment in Multiple Small Illumination Sources During Winter
Conditions
Robert M. Slade, version 1.0, 20121220
Testing can be used to demonstrate the presence of bugs, but never their
absence.
- testing aphorism
ABSTRACT
As follow-up research to the study "Risk Assessment and Failure Analysis in
Multiple Small Illumination Sources During Winter Conditions" (first published
in
2003, and available at http://catless.ncl.ac.uk/Risks/26.26.html#subj11 ), the
author has undertaken a multi-year study attempting to reduce the level and
risks
of failure in the illumination network required for celebration of the Northern
Hemisphere Mid-Winter Party Period and Gift Giving Season. (The nodes in this
network currently stand at approximately 900 sources, and a significant portion
may be noted at https://twitter.com/rslade/status/281528406917656577/photo/1 .)
Testing of nodes (also known as "bulbs") and subnets (also known as "strings")
has
been a major component of the risk reduction strategy. However, recent studies
have indicated that testing itself may be a contributing factor in node and
subnet
failures.
INTRODUCTION
In terms of risk management, it is well known that there comes a point of
diminishing returns in the process. The father of quality control, Walter
Deming,
noted that there was such a thing as too much quality assessment ( cf.
http://victoria.tc.ca/int-grps/books/techrev/bkdeming.rvw ). Despite the
greater
accuracy of assessment, very few enterprises engage in full quantitative risk
analysis, preferring the less accurate but less costly (in terms of time and
resources) qualitative risk analysis.
This study looks specifically at the testing component of the risk management
process, and notes the probability that testing may contribute to total risk or
failure.
TESTING IN THE LIGHT CYCLE
For details of the light sources and portions of the process, we refer readers
to the
earlier study ( http://catless.ncl.ac.uk/Risks/26.26.html#subj11 ). A brief
outline
of the light source cycle is in order at this point.
Towards the end of September, the female members of the household, in
preparation for upcoming events, start to ask the male members of the household
whether any purchases or other preparation is necessary. (This generally
corresponds to the initiation phase of the cycle.) The male members of the
household point out that Canadian Tire does not start selling Christmas lights
or
decorations until November. (This portion of the communication protocol is
not,
as many suppose, for information purposes, but to deflect discussion from the
fact
that the notes on necessary purchases and replacements, made last year, are
packed away with the Christmas decorations, and are therefore inaccessible.
Students of security may note that this is a good illustration of the
importance of
all three pillars of security: the confidentiality and integrity of the
information is
maintained, but availability is not.) Testing at this point in the cycle might
be
useful, but is, unfortunately, impossible.
At some point in November, the male members of the household will have run out
of excuses for not retrieving the Christmas decorations from storage. At this
point there is usually a mass retrieval of the decorations, and assessment of
any
items requiring replacement or supplement, or any perishable items which must
be
purchased each year. (This corresponds to the requirements phase.) Testing of
light nodes and subnets may be done at this point.
This retrieval/requirements phase is generally followed by a design/planning
phase.
To many researchers, it would appear that the ultimate result varies little
from
year to year, and that the design and planning is not necessary. However,
mature
researchers will note that, as one becomes, well, "more experienced" in these
matters, one notes a failing of memory as to the exact process from previous
years, and sometimes even more recent events are difficult to ...
I'm sorry, where was I? Oh, yes.
Testing and failure rectification can be undertaken during the design phase.
Some
researchers feel that this assessment point can be skipped, but experienced
researchers know that failed nodes will inevitably be discovered on the back of
the
tree in such cases.
During the implementation phase, testing tends to be somewhat informal. Since
the light nodes are being placed individually, failure of a node is generally
obvious.
However, if testing and rectification is not planned into the process,
researchers
inevitably find themselves balanced precariously on a stool at the back of the
tree,
with no replacement nodes, when a dead node or subnet is discovered.
The maintenance phase of the cycle generally runs from the first Sunday of
Advent until January 6th (Feast of the Epiphany, last of the twelve days of
Christmas). Testing at this period is by observation. Unfortunately, very
much
like testing, observation can usually tell you which nodes are shining, but not
which ones are not. As per the earlier study, it should be noted that a single
node
failure does not generally result in subnet failure, but that cumulative
failures do.
Therefore, failure to observe and rectify individual node failures frequently
result
in subnet failures at some point during this phase. Rectification following
subnet
failure at this point is extremely difficult, and usually impossible.
The termination phase of the cycle involves "undecorationing," and return of
items to storage. Testing is possible at this point of the cycle, but is made
problematic by a) fatigue, and b) haste in returning items to storage in order
to
allow for "spring cleaning."
RESULTS OF TESTING AT DIFFERENT CYCLE PHASES
Initially, this study looked at testing by observation during the maintenance
phase.
It was felt that by observation and ongoing rectification, nodes and subnets
could
be maintained, and would therefore be in good order upon retrieval the
following
year.
Unfortunately, the following year some nodes and subnets were found to be dead.
Therefore, testing at the termination phase was added. This had the advantage
of
allowing notes to be taken during rectification, so that replacements could be
purchased in advance, the year after. As previously noted, this information
was
maintained, but was not available at a time when it would be useful.
Therefore, testing was added during the requirements phase. All subnets were
tested upon retrieval, replacements were purchased (if one could fight through
the
crowds at Canadian Tire), and rectification was done prior to implementation.
During implementation phase on that study, it was found that nodes and even
subnets were still showing as failed. This led to the addition of an
additional
testing point during the design/planning phase.
During this past cycle, all nodes and subnets were tested and rectified during
the
termination phase. Upon retrieval, subnets were tested and any failures
rectified.
During planning, subnets were again tested and failures rectified. During
implemenation, provision was made for rectification within the process. So
far, in
the maintenance phase, failures have been rectified as soon as observed. (One
subnet failure was noted. The attempt to rectify it was successful, but this
is
considered anomalous.) Failure rates between testing points have been observed
as
high as 14% of total nodes.)
CONCLUSION
The results of the data collected are inescapable. Testing results in failure.
ACKNOWLEDGEMENT
This study would not have been undertaken without the encouragement and
support of Gloria J. Slade.
====================== (quote inserted randomly by Pegasus Mailer)
[email protected] [email protected] [email protected]
Great wits are sure to madness near allied. - John Dryden, 1681
victoria.tc.ca/techrev/rms.htm http://www.infosecbc.org/links
http://blogs.securiteam.com/index.php/archives/author/p1/
http://twitter.com/rslade
_______________________________________________
Fun and Misc security discussion for OT posts.
https://linuxbox.org/cgi-bin/mailman/listinfo/funsec
Note: funsec is a public and open mailing list.
