> --------------
> NSE Case Study
> --------------
>
> * Linux stars in Mission: Critical
>
> Linux is growing out of Unix's shadow and handling mission-critical tasks on
> its own. The National Stock Exchange recently replaced a Unix-based system
> for risk management with a Linux-based one and saved a packet in the
> process. Stanley Glancy reports.
>
> http://www.expresscomputeronline.com/20030825/linuxspecial02.shtml
It was interesting to read this bit in Express Computer. Took me back to
the days when we had developed it.
The PRISM system is probably the only one of its kind in the world,
which uses a cluster of computers to provide real-time risk analysis of
trades in a trading exchange, using Value-at-Risk. Before the project
was commissioned, a high-level team from the NSE visited exchanges in
the Europe and the US, including NASDAQ and NYSE, and found:
1. Some exchanges were using simple upper-limit-lower-limit software to
do real-time risk checking of buyers and sellers for each trade. The
software worked in real time all right, but all it did was calculate
some primitive risk attribute of each portfolio and compare it
with the high and low limits. Any economist will tell you that this
super-simplistic risk limiting makes almost no sense in the context
of the dynamics of a busy market. I don't know enough econometrics
to even begin to comment on this.
2. Other exchanges were using mathematical models like VaR for risk
analysis, but not in real time. They'd do the number-crunching at
the end-of-day, and any trader who had crossed risk limits would be
debarred from trading the next morning. Therefore, intra-day, a
rogue trader could bring down part of the exchange by his crazy risk
exposure.
The NSE wanted to go the whole hog, and build a real-time risk analysis
engine which would use a sophisticated VaR model for risk analysis. This
VaR model would be hand-tuned to the parameters of the NSE, based on
NSE's own historical data about scripts, volatility of the market, etc.
This, I have been assured, is a complex area, where current research
is going on, and I believe the porting of the VaR model to the NSE data
itself is worthy of a PhD or two in economics.
This work was done by a team led by Ajay Shah and Susan Thomas, both
faculty at the IGIDR. They have a long association with the NSE; they have
made key contributions to the econometrics theory and its application
in deciding the principles of the NSE Nifty index. (I could be wrong,
but I'm told that the formulation of the BSE Sensex is not based on
any hard econometrics theory.) In fact Susan, I believe, thought up the
name "Nifty" for the NSE-Fifty index. :) The IGIDR team undertook the
responsibility of actually converting the VaR math into a well-written,
debugged, C function. Ajay and Susan have done _enormous_ amounts of
coding in C, awk and Perl, so they can deliver such efforts better
than the average economist. Ajay often runs huge numerical computation
jobs on his Linux laptop, where the code is written in Perl, crunches
gigabytes of actual data all kept in plain ASCII files, and takes actual
days of computation. :)
Everything else in PRISM was our responsibility. We worked on:
(a) the overall software architecture,
(b) the partitioning of computation into the various components in
the cluster,
(c) the design of the front-end (it's a GUI Java application; we
needed to impress non-geeks with PRISM by _showing_ them something
happening on the screen),
(d) the fault tolerance architecture
(e) the throughput measurement and scalability
(f) the choice of technology (we studied PVM and MPI)
(g) and of course, writing and debugging lots of C code. :)
among other things.
There were other parties other than IGIDR and Starcom too in the picture,
and most of the coding for the Java GUI front-end for instance was done by
programmers not working in Starcom. There was no employee of NSE or NSE.IT
involved in the design or development of PRISM, IIRC. I remember giving
demos to Ravi Narain, then DyMD of NSE (now MD), and Satish Naralkar
(CEO of NSE.IT). Both were very encouraging and supportive. After all,
they were "the customer." :) And we were all aware that in our small way,
we were creating history. The fact that it was also on an open source
OS was icing on the cake. :)
Another high point of the project was taking our code to C-DAC Pune to
get it benchmarked on the PARAM. We interacted with Dr.Medha Dhurandhar
there, and her team assisted our team in porting our code onto their
latest PARAM. In terms of CPU power, each PARAM CPU was woefully behind
the times (some 330MHz SPARC CPU, I think) even for the standards of 1999
and 2000, but connect them together, and they could really sing. We found
that the PARAM architecture running on normal Fast Ethernet couldn't
add much value to our measured Intel performance in IGIDR's PRISM Lab
with the same Fast Ethernet technology. But shift the PARAM to Myrinet,
and that performance really shot up. Therefore, at least in the case of
the PARAM, it was clear that the slow CPUs were not really holding the
system back. This was also partly true because of the nature of PRISM.
The application is so highly parallelisable that you can throw more CPUs
at the problem and compensate for their individual slowness.
PRISM went live in 2000, and has stayed live since then. After we had
delivered the initial version and proved its stability, the NSE.IT stepped
in and took ownership of the system. They have made many improvements to
the code, I believe, and added new algorithms of risk analysis, etc. But
some of the key technology and architecture decisions have remained,
(I think!):
(a) a master-slave architecture, with lots of scalability for the slaves
but no parallelism for master
(b) Linux for the slaves. (We had set up the master on a DEC Alpha
quad-processor box; the Express Computer mentions Shenoy saying that
the master is on an Intel box now.)
(c) the GUI front-end implemented as a Java application, talking to the
master through a specially-built loosely-coupled protocol designed
not to slow down the actual risk analysis while GUI events happen.
(d) MPI for the message-passing backbone for the cluster application
(e) VaR for the core risk analysis (other algos have been added, I'm
told, but VaR remains.)
(f) fault tolerance upto order K-1 for failure of slaves (assuming that
you are running with K slaves), but no fault tolerance for the
master.
The PRISM system remains a show-case application which is proudly
demonstrated to every official visitor to the exchange, I'm told. It also
continues to function, scaling up its throughput smoothly, indicating
that our initial architecture was valid. This is no surprise, really;
we had tested it and shown that it could handle trading volumes ten times
the peak throughput experienced by the NSE in those days, by using just
a dozen dual-processor Intel boxes for slaves.
In 2001, I was asked to speak at the Singapore Linux Conference, and Liz
Coolbaugh from the LWN was covering the event. (She was also a speaker.)
She covered my talk on PRISM; it's still present in the LWN archives at
http://lwn.net/2001/features/Singapore/prism.php3
(Liz is no more with LWN. We miss her; she is a very lovely person.)
PRISM was a fun project. :)
By the way, Ajay is currently on deputation from the IGIDR to the
Ministry of Finance of the Government of India; he stays in Delhi now.
He's been using Unix and Linux for at least fifteen years now, and is
familiar with the Unix "way" of doing things. For instance, he uses
Perl code for econometrics computations, and uses LaTeX fluently for
all formatted documentation. (Thanks to the outbreak of the SoBig worm,
I discovered that not all Linux veterans use Linux on their desktops. A
lot of members of the LDP, who have done a lot of work for Linux, use
Windows desktops; some of them have been hit by SoBig...) And to top
it all, Ajay is a very sharp brain and a good speaker. I feel ILUGD
should try to catch hold of him for talks, presentations, etc. Check
his homepage at http://www.mayin.org/~ajayshah/
Shuvam
_______________________________________________
ilugd mailing list
[EMAIL PROTECTED]
http://frodo.hserus.net/mailman/listinfo/ilugd
