Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Scott Ford]

Shmuel:
 
I work the same way, we have to test, no opinions, being a ISV. We support 
three security subsystems...RACF,ACF2 and Top-Secret. Even as a Sysprog on 
VM/VSE and MVS I always tested. To me personally, crazy not to, if I wanted to 
stay employed.
Secondly, I have a responsibility to my employer to do my jo2 to the best of my 
ability and then some...but thats me

Scott J Ford
Software Engineer
http://www.identityforge.com/
 
 


 From: Shmuel Metz (Seymour J.) 
To: IBM-MAIN@LISTSERV.UA.EDU 
Sent: Tuesday, July 3, 2012 8:14 AM
Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking 
meltdown
  
In
,
on 07/02/2012
   at 05:33 PM, John Gilmore  said:

>John Reid has reminded us all of a generic weakness in the way we

TINW.

>do things.

I don't know about you, but when I install a new system I test it,
have an SME test it or both.

>Comprehensive IVPs have all but disappeared;

That's a problem for a new system, but for an upgrade you should
already have a regression test.

-- 
     Shmuel (Seymour J.) Metz, SysProg and JOAT
     Atid/2        <http://patriot.net/~shmuel>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Scott Ford]

Shmuel,

So have I ..competent people usually are on the same technical page, for sure

Scott ford
www.identityforge.com

On Jul 2, 2012, at 9:13 AM, "Shmuel Metz (Seymour J.)"  
wrote:

> In <24ed21e9-0ebc-42b6-af26-5e07aa72c...@yahoo.com>, on 07/02/2012
>   at 12:51 AM, Scott Ford  said:
> 
>> This industry is tough enough without a language barrier. Add the
>> language barrier it gets down right ugly.
> 
> I've worked with competent people with a language barrier. I've worked
> with incompetent people who spoke flawless English. I'll take the
> competent people with a language barrier, TYVM.
> 
> -- 
> Shmuel (Seymour J.) Metz, SysProg and JOAT
> Atid/2
> We don't care. We don't have to care, we're Congress.
> (S877: The Shut up and Eat Your spam act of 2003)
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Shmuel Metz (Seymour J.)]

In
,
on 07/02/2012
   at 05:33 PM, John Gilmore  said:

>John Reid has reminded us all of a generic weakness in the way we

TINW.

>do things.

I don't know about you, but when I install a new system I test it,
have an SME test it or both.

>Comprehensive IVPs have all but disappeared;

That's a problem for a new system, but for an upgrade you should
already have a regression test.

-- 
 Shmuel (Seymour J.) Metz, SysProg and JOAT
 Atid/2
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Shmuel Metz (Seymour J.)]

In
,
on 07/01/2012
   at 10:50 PM, John Gilmore  said:

>Regrettablly, incoherent questions are not limited to "foreign
>speakers".

>Moreover, we have a number of "foreign speakers" among our most
>useful contributors.

Indeed, on both counts. No country has a monopoly on intelligence, and
no country has a monopoly on incompetence.

-- 
 Shmuel (Seymour J.) Metz, SysProg and JOAT
 Atid/2
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Shmuel Metz (Seymour J.)]

In <24ed21e9-0ebc-42b6-af26-5e07aa72c...@yahoo.com>, on 07/02/2012
   at 12:51 AM, Scott Ford  said:

>This industry is tough enough without a language barrier. Add the
>language barrier it gets down right ugly.

I've worked with competent people with a language barrier. I've worked
with incompetent people who spoke flawless English. I'll take the
competent people with a language barrier, TYVM.

-- 
 Shmuel (Seymour J.) Metz, SysProg and JOAT
 Atid/2
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Ed Gould]

Mark:

I spoke to the individuals that I have interviewed.

Ed

On Jul 2, 2012, at 7:54 AM, Mark Zelden wrote:

On Sun, 1 Jul 2012 09:33:05 -0500, Ed Gould  
 wrote:



Shmuel:

Well if it was outsourced I wouldn't be surprised to much.
I suspect that these people sell themselves as experts in everything
CA-7 or MVS you name it. I have interviewed two foreign speaking
individuals and it turns out they read the manual and do not have a
clue beyond that.



Talk about a blanket generalization!Ed, when was the last time you
did any real work in MVS or worked with or in a company that
off shored any of their work?

And I don't know the specifics of this incident and haven't been  
following

it closely, but based on the CA-7 PTFs I saw released right afterwards
I would say the problem was related to the attempted upgrade (highly
doubtful that it was operators that attempted it) and backout coupled
with a CA-7 software issue that RBS couldn't have foreseen.

Mark
--
Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS
mailto:m...@mzelden.com
Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html
Systems Programming expert at http:// 
expertanswercenter.techtarget.com/


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Scott Ford]

John,

I also worked with Europeans, while living in Europe, also Mexico, including 
the Swiss.  In my old age, 62 in August, I have concluded that the new IT world 
order,  that's what my boss alls it, the solution to testing , is call the 
vendor. We get ton of calls, usually because ppl don't read the manuals. Many 
of them i feel haven't  had the fortune of learning z/OS from the early days. I 
learned CICS many moons ago with the source assemblies and PLMs. Unfortunately, 
the have shall we say disappeared. 

Scott ford
www.identityforge.com

On Jul 2, 2012, at 7:43 PM, John Gilmore  wrote:

> 'Sysprog' is unfortunately an ambiguous term.  There are those who
> build software for ISVs or, of course, IBM; and there are those who
> keep a z/OS shop running.
> 
> As a part of an attempt to help in the selection of a new manager for
> a systems programming group in a European insurance company I recently
> spent 3 half days, one of them with each of three of the latter.  They
> installed things; did maintenance using SMP/E, etc., etc.  They did
> damn little testing, escept in the sense that the systematic
> correction of errors in their own work is testing.
> 
> Moreover, it became clear that none of them knew or could know enough
> about all of the components they were working with to test them
> independently.  (They were competent people, but they were not
> polymaths.)  This melancholy conclusion led me to the notion that IVPs
> and MVPs, most of them necessarily supplied by vendors, needed to be
> widely available.
> 
> These people have other needs too.  They seem, for example, to have
> very little sense of the ways in which notionally very different kinds
> of IBM systems are almost all much alike in some ways.  They cannot
> make plausible, often (but not of course always) confirmed inferences
> about how things work using this kind of knowledge.  Moreover, I am
> not sure I know how to communicate/teach these skills outside of
> essentially one-on-one, apprenticeship situations.
> 
> John Gilmore, Ashland, MA 01721 - USA
> 
> On 7/2/12, Scott Ford  wrote:
>> John,
>> 
>> I don't know what sysprogs you have been around, but I can tell you i test,
>> test, test , test...always have and always will. Part of the problem are the
>> times and education and experience
>> But I also think its how you were initially trained.
>> 
>> Scott ford
>> www.identityforge.com
>> 
>> On Jul 2, 2012, at 5:33 PM, John Gilmore  wrote:
>> 
>>> John Reid has reminded us all of a generic weakness in the way we do
>>> things.  Sysprogs install new systems and maintain them, but they do
>>> not usually test them.  They may indeed have only the vaguest notions
>>> of what some of them do.
>>> 
>>> Comprehensive IVPs have all but disappeared; but it it time, I think,
>>> to reintroduce them.  Moreover, there is a need for MVPs (Maintenance
>>> Verification Procedures) too.
>>> 
>>> If they were always available sysprogs could properly be given the
>>> responsibility for using them and exami ng their outputs.
>>> 
>>> John Gilmore, Ashland, MA 01721 - USA
>>> 
>>> On 7/2/12, John Blythe Reid  wrote:
 From what I've read, and it sounds quite as you would expect, the
 systems
 programmers who applied the maintenance to CA-7 were based in the main
 Edinburgh data centre whereas the team responsible for job scheduling
 using
 CA-7 were based in Hyderabad. As we all know, when a critical job abends
 during the middle of the night with a tight batch window, things become
 quite stressful. And that rising stress level can lead to someone
 pressing
 the wrong key. Normally the consequences are not as disastrous as this
 one's been.
 
 John.
 
 On 2 July 2012 14:54, Mark Zelden  wrote:
 
> On Sun, 1 Jul 2012 09:33:05 -0500, Ed Gould 
> wrote:
> 
>> Shmuel:
>> 
>> Well if it was outsourced I wouldn't be surprised to much.
>> I suspect that these people sell themselves as experts in everything
>> CA-7 or MVS you name it. I have interviewed two foreign speaking
>> individuals and it turns out they read the manual and do not have a
>> clue beyond that.
>> 
> 
> Talk about a blanket generalization!Ed, when was the last time you
> did any real work in MVS or worked with or in a company that
> off shored any of their work?
> 
> And I don't know the specifics of this incident and haven't been
> following
> it closely, but based on the CA-7 PTFs I saw released right afterwards
> I would say the problem was related to the attempted upgrade (highly
> doubtful that it was operators that attempted it) and backout coupled
> with a CA-7 software issue that RBS couldn't have foreseen.
> 
> Mark
> --
> Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS
> mailto:m...@mzelden.com
> Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html
> Systems Progr

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[John Gilmore]

'Sysprog' is unfortunately an ambiguous term.  There are those who
build software for ISVs or, of course, IBM; and there are those who
keep a z/OS shop running.

As a part of an attempt to help in the selection of a new manager for
a systems programming group in a European insurance company I recently
spent 3 half days, one of them with each of three of the latter.  They
installed things; did maintenance using SMP/E, etc., etc.  They did
damn little testing, escept in the sense that the systematic
correction of errors in their own work is testing.

Moreover, it became clear that none of them knew or could know enough
about all of the components they were working with to test them
independently.  (They were competent people, but they were not
polymaths.)  This melancholy conclusion led me to the notion that IVPs
and MVPs, most of them necessarily supplied by vendors, needed to be
widely available.

These people have other needs too.  They seem, for example, to have
very little sense of the ways in which notionally very different kinds
of IBM systems are almost all much alike in some ways.  They cannot
make plausible, often (but not of course always) confirmed inferences
about how things work using this kind of knowledge.  Moreover, I am
not sure I know how to communicate/teach these skills outside of
essentially one-on-one, apprenticeship situations.

John Gilmore, Ashland, MA 01721 - USA

On 7/2/12, Scott Ford  wrote:
> John,
>
> I don't know what sysprogs you have been around, but I can tell you i test,
> test, test , test...always have and always will. Part of the problem are the
> times and education and experience
> But I also think its how you were initially trained.
>
> Scott ford
> www.identityforge.com
>
> On Jul 2, 2012, at 5:33 PM, John Gilmore  wrote:
>
>> John Reid has reminded us all of a generic weakness in the way we do
>> things.  Sysprogs install new systems and maintain them, but they do
>> not usually test them.  They may indeed have only the vaguest notions
>> of what some of them do.
>>
>> Comprehensive IVPs have all but disappeared; but it it time, I think,
>> to reintroduce them.  Moreover, there is a need for MVPs (Maintenance
>> Verification Procedures) too.
>>
>> If they were always available sysprogs could properly be given the
>> responsibility for using them and exami ng their outputs.
>>
>> John Gilmore, Ashland, MA 01721 - USA
>>
>> On 7/2/12, John Blythe Reid  wrote:
>>> From what I've read, and it sounds quite as you would expect, the
>>> systems
>>> programmers who applied the maintenance to CA-7 were based in the main
>>> Edinburgh data centre whereas the team responsible for job scheduling
>>> using
>>> CA-7 were based in Hyderabad. As we all know, when a critical job abends
>>> during the middle of the night with a tight batch window, things become
>>> quite stressful. And that rising stress level can lead to someone
>>> pressing
>>> the wrong key. Normally the consequences are not as disastrous as this
>>> one's been.
>>>
>>> John.
>>>
>>> On 2 July 2012 14:54, Mark Zelden  wrote:
>>>
 On Sun, 1 Jul 2012 09:33:05 -0500, Ed Gould 
 wrote:

> Shmuel:
>
> Well if it was outsourced I wouldn't be surprised to much.
> I suspect that these people sell themselves as experts in everything
> CA-7 or MVS you name it. I have interviewed two foreign speaking
> individuals and it turns out they read the manual and do not have a
> clue beyond that.
>

 Talk about a blanket generalization!Ed, when was the last time you
 did any real work in MVS or worked with or in a company that
 off shored any of their work?

 And I don't know the specifics of this incident and haven't been
 following
 it closely, but based on the CA-7 PTFs I saw released right afterwards
 I would say the problem was related to the attempted upgrade (highly
 doubtful that it was operators that attempted it) and backout coupled
 with a CA-7 software issue that RBS couldn't have foreseen.

 Mark
 --
 Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS
 mailto:m...@mzelden.com
 Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html
 Systems Programming expert at http://expertanswercenter.techtarget.com/

 --
 For IBM-MAIN subscribe / signoff / archive access instructions,
 send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

>>>
>>>
>>>
>>> --
>>> John Blythe Reid,
>>> Técnico de Sistemas de z/OS y de Sistemas Transaccionales,
>>> Barcelona,
>>> España.
>>>
>>> --
>>> For IBM-MAIN subscribe / signoff / archive access instructions,
>>> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>>>
>>
>> --
>> For IBM-MAIN subscribe /

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Scott Ford]

John,

I don't know what sysprogs you have been around, but I can tell you i test, 
test, test , test...always have and always will. Part of the problem are the 
times and education and experience
But I also think its how you were initially trained.

Scott ford
www.identityforge.com

On Jul 2, 2012, at 5:33 PM, John Gilmore  wrote:

> John Reid has reminded us all of a generic weakness in the way we do
> things.  Sysprogs install new systems and maintain them, but they do
> not usually test them.  They may indeed have only the vaguest notions
> of what some of them do.
> 
> Comprehensive IVPs have all but disappeared; but it it time, I think,
> to reintroduce them.  Moreover, there is a need for MVPs (Maintenance
> Verification Procedures) too.
> 
> If they were always available sysprogs could properly be given the
> responsibility for using them and exami ng their outputs.
> 
> John Gilmore, Ashland, MA 01721 - USA
> 
> On 7/2/12, John Blythe Reid  wrote:
>> From what I've read, and it sounds quite as you would expect, the systems
>> programmers who applied the maintenance to CA-7 were based in the main
>> Edinburgh data centre whereas the team responsible for job scheduling using
>> CA-7 were based in Hyderabad. As we all know, when a critical job abends
>> during the middle of the night with a tight batch window, things become
>> quite stressful. And that rising stress level can lead to someone pressing
>> the wrong key. Normally the consequences are not as disastrous as this
>> one's been.
>> 
>> John.
>> 
>> On 2 July 2012 14:54, Mark Zelden  wrote:
>> 
>>> On Sun, 1 Jul 2012 09:33:05 -0500, Ed Gould 
>>> wrote:
>>> 
 Shmuel:
 
 Well if it was outsourced I wouldn't be surprised to much.
 I suspect that these people sell themselves as experts in everything
 CA-7 or MVS you name it. I have interviewed two foreign speaking
 individuals and it turns out they read the manual and do not have a
 clue beyond that.
 
>>> 
>>> Talk about a blanket generalization!Ed, when was the last time you
>>> did any real work in MVS or worked with or in a company that
>>> off shored any of their work?
>>> 
>>> And I don't know the specifics of this incident and haven't been
>>> following
>>> it closely, but based on the CA-7 PTFs I saw released right afterwards
>>> I would say the problem was related to the attempted upgrade (highly
>>> doubtful that it was operators that attempted it) and backout coupled
>>> with a CA-7 software issue that RBS couldn't have foreseen.
>>> 
>>> Mark
>>> --
>>> Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS
>>> mailto:m...@mzelden.com
>>> Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html
>>> Systems Programming expert at http://expertanswercenter.techtarget.com/
>>> 
>>> --
>>> For IBM-MAIN subscribe / signoff / archive access instructions,
>>> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>>> 
>> 
>> 
>> 
>> --
>> John Blythe Reid,
>> Técnico de Sistemas de z/OS y de Sistemas Transaccionales,
>> Barcelona,
>> España.
>> 
>> --
>> For IBM-MAIN subscribe / signoff / archive access instructions,
>> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>> 
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[John Gilmore]

John Reid has reminded us all of a generic weakness in the way we do
things.  Sysprogs install new systems and maintain them, but they do
not usually test them.  They may indeed have only the vaguest notions
of what some of them do.

Comprehensive IVPs have all but disappeared; but it it time, I think,
to reintroduce them.  Moreover, there is a need for MVPs (Maintenance
Verification Procedures) too.

If they were always available sysprogs could properly be given the
responsibility for using them and exami ng their outputs.

John Gilmore, Ashland, MA 01721 - USA

On 7/2/12, John Blythe Reid  wrote:
> From what I've read, and it sounds quite as you would expect, the systems
> programmers who applied the maintenance to CA-7 were based in the main
> Edinburgh data centre whereas the team responsible for job scheduling using
> CA-7 were based in Hyderabad. As we all know, when a critical job abends
> during the middle of the night with a tight batch window, things become
> quite stressful. And that rising stress level can lead to someone pressing
> the wrong key. Normally the consequences are not as disastrous as this
> one's been.
>
> John.
>
> On 2 July 2012 14:54, Mark Zelden  wrote:
>
>> On Sun, 1 Jul 2012 09:33:05 -0500, Ed Gould 
>> wrote:
>>
>> >Shmuel:
>> >
>> >Well if it was outsourced I wouldn't be surprised to much.
>> >I suspect that these people sell themselves as experts in everything
>> >CA-7 or MVS you name it. I have interviewed two foreign speaking
>> >individuals and it turns out they read the manual and do not have a
>> >clue beyond that.
>> >
>>
>> Talk about a blanket generalization!Ed, when was the last time you
>> did any real work in MVS or worked with or in a company that
>> off shored any of their work?
>>
>> And I don't know the specifics of this incident and haven't been
>> following
>> it closely, but based on the CA-7 PTFs I saw released right afterwards
>> I would say the problem was related to the attempted upgrade (highly
>> doubtful that it was operators that attempted it) and backout coupled
>> with a CA-7 software issue that RBS couldn't have foreseen.
>>
>> Mark
>> --
>> Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS
>> mailto:m...@mzelden.com
>> Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html
>> Systems Programming expert at http://expertanswercenter.techtarget.com/
>>
>> --
>> For IBM-MAIN subscribe / signoff / archive access instructions,
>> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>>
>
>
>
> --
> John Blythe Reid,
> Técnico de Sistemas de z/OS y de Sistemas Transaccionales,
> Barcelona,
> España.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[John Blythe Reid]

>From what I've read, and it sounds quite as you would expect, the systems
programmers who applied the maintenance to CA-7 were based in the main
Edinburgh data centre whereas the team responsible for job scheduling using
CA-7 were based in Hyderabad. As we all know, when a critical job abends
during the middle of the night with a tight batch window, things become
quite stressful. And that rising stress level can lead to someone pressing
the wrong key. Normally the consequences are not as disastrous as this
one's been.

John.

On 2 July 2012 14:54, Mark Zelden  wrote:

> On Sun, 1 Jul 2012 09:33:05 -0500, Ed Gould 
> wrote:
>
> >Shmuel:
> >
> >Well if it was outsourced I wouldn't be surprised to much.
> >I suspect that these people sell themselves as experts in everything
> >CA-7 or MVS you name it. I have interviewed two foreign speaking
> >individuals and it turns out they read the manual and do not have a
> >clue beyond that.
> >
>
> Talk about a blanket generalization!Ed, when was the last time you
> did any real work in MVS or worked with or in a company that
> off shored any of their work?
>
> And I don't know the specifics of this incident and haven't been following
> it closely, but based on the CA-7 PTFs I saw released right afterwards
> I would say the problem was related to the attempted upgrade (highly
> doubtful that it was operators that attempted it) and backout coupled
> with a CA-7 software issue that RBS couldn't have foreseen.
>
> Mark
> --
> Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS
> mailto:m...@mzelden.com
> Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html
> Systems Programming expert at http://expertanswercenter.techtarget.com/
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>



-- 
John Blythe Reid,
Técnico de Sistemas de z/OS y de Sistemas Transaccionales,
Barcelona,
España.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[McKown, John]

> -Original Message-
> From: IBM Mainframe Discussion List 
> [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Mark Zelden
> Sent: Monday, July 02, 2012 7:55 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led 
> to banking meltdown
> 
> On Sun, 1 Jul 2012 09:33:05 -0500, Ed Gould 
>  wrote:
> 
> >Shmuel:
> >
> >Well if it was outsourced I wouldn't be surprised to much.
> >I suspect that these people sell themselves as experts in everything
> >CA-7 or MVS you name it. I have interviewed two foreign speaking
> >individuals and it turns out they read the manual and do not have a
> >clue beyond that.
> >
> 
> Talk about a blanket generalization!Ed, when was the last 
> time you 
> did any real work in MVS or worked with or in a company that
> off shored any of their work?
> 
> And I don't know the specifics of this incident and haven't 
> been following
> it closely, but based on the CA-7 PTFs I saw released right afterwards
> I would say the problem was related to the attempted upgrade (highly
> doubtful that it was operators that attempted it) and backout coupled
> with a CA-7 software issue that RBS couldn't have foreseen. 
> 
> Mark
> --
> Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS   

I do know that we had a CA7 outage recently at IPL time, an S0C6. I am lucky, 
my manager is a CA7 expert and even knows some of the CA people personally. We 
did not have any corruption, just an ABEND. We just applied a HIPER to fix it. 
We were "bleeding edge" on CA7 maintenance.

-- 
John McKown 
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets(r)

9151 Boulevard 26 * N. Richland Hills * TX 76010
(817) 255-3225 phone * 
john.mck...@healthmarkets.com * www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or 
proprietary information. If you are not the intended recipient, please contact 
the sender by reply e-mail and destroy all copies of the original message. 
HealthMarkets(r) is the brand name for products underwritten and issued by the 
insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance 
Company(r), Mid-West National Life Insurance Company of TennesseeSM and The 
MEGA Life and Health Insurance Company.SM

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Mark Zelden]

On Sun, 1 Jul 2012 09:33:05 -0500, Ed Gould  wrote:

>Shmuel:
>
>Well if it was outsourced I wouldn't be surprised to much.
>I suspect that these people sell themselves as experts in everything
>CA-7 or MVS you name it. I have interviewed two foreign speaking
>individuals and it turns out they read the manual and do not have a
>clue beyond that.
>

Talk about a blanket generalization!Ed, when was the last time you 
did any real work in MVS or worked with or in a company that
off shored any of their work?

And I don't know the specifics of this incident and haven't been following
it closely, but based on the CA-7 PTFs I saw released right afterwards
I would say the problem was related to the attempted upgrade (highly
doubtful that it was operators that attempted it) and backout coupled
with a CA-7 software issue that RBS couldn't have foreseen. 

Mark
--
Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS   
mailto:m...@mzelden.com
Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html 
Systems Programming expert at http://expertanswercenter.techtarget.com/

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Scott Ford]

This industry is tough enough without a language barrier. Add the language 
barrier it gets down right ugly. Plus add cultural differences...man, plus lack 
of experience

Scott ford
www.identityforge.com

On Jul 1, 2012, at 11:56 PM, zMan  wrote:

> On Sun, Jul 1, 2012 at 10:50 PM, John Gilmore  wrote:
> 
>> Regrettablly, incoherent questions are not limited to "foreign speakers".
>> 
>> Moreover, we have a number of "foreign speakers" among our most useful
>> contributors.
>> 
>> There was always the danger that it would do so, and this thread has
>> now taken a disagreeably xenophobic turn.  I suggest that we kill it
>> off before it becomes even more offensive.
>> 
> 
> Good point. I really wasn't thinking in those terms -- I was thinking of
> the questions that are so basic that people usually ask, "Is this for a
> school project?"
> 
> But yeah, it is at risk of turning into anti-foreigner rants. I'm out.
> -- 
> zMan -- "I've got a mainframe and I'm not afraid to use it"
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[zMan]

On Sun, Jul 1, 2012 at 10:50 PM, John Gilmore  wrote:

> Regrettablly, incoherent questions are not limited to "foreign speakers".
>
> Moreover, we have a number of "foreign speakers" among our most useful
> contributors.
>
> There was always the danger that it would do so, and this thread has
> now taken a disagreeably xenophobic turn.  I suggest that we kill it
> off before it becomes even more offensive.
>

Good point. I really wasn't thinking in those terms -- I was thinking of
the questions that are so basic that people usually ask, "Is this for a
school project?"

But yeah, it is at risk of turning into anti-foreigner rants. I'm out.
-- 
zMan -- "I've got a mainframe and I'm not afraid to use it"

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Anne & Lynn Wheeler]

re:
http://www.garlic.com/~lynn/2012i.html#46 'Inexperienced' RBS tech operative's 
blunder led to banking meltdown
http://www.garlic.com/~lynn/2012i.html#58 'Inexperienced' RBS tech operative's 
blunder led to banking meltdown

it isn't RBS only problem:

Another Domino Falls in the LIBOR Banking Scam: Royal Bank of Scotland
http://www.rollingstone.com/politics/blogs/taibblog/another-domino-falls-in-the-libor-banking-scam-royal-bank-of-scotland-20120629ï

part of this:

A Huge Break in the LIBOR Banking Investigation
http://www.rollingstone.com/politics/blogs/taibblog/a-huge-break-in-the-libor-banking-investigation-20120628

Naked emperors, holy cows and Libor
http://www.atimes.com/atimes/Global_Economy/NF30Dj02.html

$700T worldwide:
http://www.atimes.com/atimes/Global_Economy/ML14Dj02.html

much of it in the US:
http://demonocracy.info/infographics/usa/derivatives/bank_exposure.html

a few more, now will anybody go to jail??

Banks face lawsuits worth billions over Libor scam
http://www.independent.co.uk/news/business/news/banks-face-lawsuits-worth-billions-over-libor-scam-7902918.html
U.K. Serious Fraud Office Opted Against Libor Criminal Probe: FT
http://www.bloomberg.com/news/2012-06-30/u-k-serious-fraud-office-opted-against-libor-criminal-probe-ft.html
Barclays LIBOR Market Manipulation Fraud To Boost Profits and Mask Insolvency, 
RBS, HSBC and Lloyds to Follow :: The Market Oracle ::
http://www.marketoracle.co.uk/Article35374.html
Banking scandal: how document trail reveals global scam
http://www.guardian.co.uk/business/2012/jun/30/banking-scandal-barclays-lawsuits-libor?newsfeed=trueï»

not directly ... other recent posts on derivatives
http://www.garlic.com/~lynn/2012h.html#45 Monopoly/ Cartons of Punch Cards
http://www.garlic.com/~lynn/2012i.html#17 Monopoly/ Cartons of Punch Cards
http://www.garlic.com/~lynn/2012i.html#29 Monopoly/ Cartons of Punch Cards

-- 
virtualization experience starting Jan1968, online at home since Mar1970

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Anthony Thompson]

War story:

There was a certain major US-based mainframe software company (which I will not 
name) who decided to outsource the maintenance/development of one of their 
software products to an external company (not US-based, and once again, for 
fear of offending the overly-sensitive, I will not name the country). This 
product was partly written in some serious assembler code. Just before the 
contract was awarded, the in-house developers of the product thought it would 
be a good idea to do some due diligence, and interview the prospective future 
developers.

One of the questions they asked was: What is a supervisor call?

The answer: That's when we pick up the phone and call our boss.

When pressed, it became apparent they didn't know what a SVC was.

They didn't get the contract, despite it being very late in the process, it was 
awarded to the small company that I then worked for.

Unfortunately, some managers will take decisions based on their perceived 
financial value and little else. The risk analysis seems secondary, further, 
they don't seem to understand the value of decades of in-house expertise and 
knowledge. Such is business. Or, as Roger Waters (ex-Pink Floyd) said: "Don't 
be afraid, it's only business."

Ant.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[John Gilmore]

Regrettablly, incoherent questions are not limited to "foreign speakers".

Moreover, we have a number of "foreign speakers" among our most useful
contributors.

There was always the danger that it would do so, and this thread has
now taken a disagreeably xenophobic turn.  I suggest that we kill it
off before it becomes even more offensive.

John Gilmore, Ashland, MA 01721 -USA

On 7/1/12, zMan  wrote:
> On Sun, Jul 1, 2012 at 10:33 AM, Ed Gould  wrote:
>
>> Shmuel:
>>
>> Well if it was outsourced I wouldn't be surprised to much.
>> I suspect that these people sell themselves as experts in everything CA-7
>> or MVS you name it. I have interviewed two foreign speaking individuals
>> and
>> it turns out they read the manual and do not have a clue beyond that.
>
>
> And then they get on IBM-MAIN and ask incoherent questions...
> --
> zMan -- "I've got a mainframe and I'm not afraid to use it"
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[zMan]

On Sun, Jul 1, 2012 at 10:33 AM, Ed Gould  wrote:

> Shmuel:
>
> Well if it was outsourced I wouldn't be surprised to much.
> I suspect that these people sell themselves as experts in everything CA-7
> or MVS you name it. I have interviewed two foreign speaking individuals and
> it turns out they read the manual and do not have a clue beyond that.


And then they get on IBM-MAIN and ask incoherent questions...
-- 
zMan -- "I've got a mainframe and I'm not afraid to use it"

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Shmuel Metz (Seymour J.)]

In ,
on 06/28/2012
   at 07:44 AM, "McKown, John"  said:

>My personal desire would be something akin to ISPF available from a
>z/OS UNIX shell

If IBM were willing, the ISPF WSA would be a good start.

>and maybe the TSO dependant parts would work properly in an 
>IKJTSOEV environment.

Why bother? Run the z/OS side in a batch job running IKJEFT01.

>And for simple editing, I prefer vim or gvim.

De gustibus. I prefer SPF for editing PC files. but at least you're
using vim and gvim rather than vi ;-)

-- 
 Shmuel (Seymour J.) Metz, SysProg and JOAT
 Atid/2
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Ed Gould]

Shmuel:

Well if it was outsourced I wouldn't be surprised to much.
I suspect that these people sell themselves as experts in everything  
CA-7 or MVS you name it. I have interviewed two foreign speaking  
individuals and it turns out they read the manual and do not have a  
clue beyond that.


Ed

On Jul 1, 2012, at 9:04 AM, Shmuel Metz (Seymour J.) wrote:


In <047067a6-a5c0-4f92-8963-1bd539013...@comcast.net>, on 06/26/2012
   at 11:37 PM, Ed Gould  said:


Subject: 'Inexperienced' RBS tech operative's blunder led to banking
meltdown


An *operator* was upgrading CA-7? Tell me it isn't true!

--
 Shmuel (Seymour J.) Metz, SysProg and JOAT
 Atid/2
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Shmuel Metz (Seymour J.)]

In <047067a6-a5c0-4f92-8963-1bd539013...@comcast.net>, on 06/26/2012
   at 11:37 PM, Ed Gould  said:

>Subject: 'Inexperienced' RBS tech operative's blunder led to banking
>meltdown

An *operator* was upgrading CA-7? Tell me it isn't true! 

-- 
 Shmuel (Seymour J.) Metz, SysProg and JOAT
 Atid/2
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Scott Ford]

Kirk,

Absolutely ugly in regression testing...Or found Friday at 5 pm

Scott ford
www.identityforge.com

On Jun 28, 2012, at 1:12 PM, Kirk Talman  wrote:

> well said
> 
> not supposed to exist, but ...
> 
> especially ugly when the problem is regression testing
> 
> IBM Mainframe Discussion List  wrote on 
> 06/27/2012 05:04:52 PM:
> 
>> From: Scott Ford 
>> 
>> Sounded like production testing..if it works it's productions if notits 
> a test
>> 
>> Scott ford
>> www.identityforge.com
> 
> 
> -
> The information contained in this communication (including any
> attachments hereto) is confidential and is intended solely for the
> personal and confidential use of the individual or entity to whom
> it is addressed. If the reader of this message is not the intended
> recipient or an agent responsible for delivering it to the intended
> recipient, you are hereby notified that you have received this
> communication in error and that any review, dissemination, copying,
> or unauthorized use of this information, or the taking of any
> action in reliance on the contents of this information is strictly
> prohibited. If you have received this communication in error,
> please notify us immediately by e-mail, and delete the original
> message. Thank you 
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Scott Ford]

Ed,

I worked at a bank in manhattan and they had a encrypted line to the Fed , if 
down fines were $2 a min..seen a lot ppl getting anxious over a down comm. 
line, but understandable

Scott ford
www.identityforge.com

On Jun 28, 2012, at 7:16 PM, Ed Gould  wrote:

> On Jun 28, 2012, at 4:00 PM, Anne & Lynn Wheeler wrote:
>> 
>> ibm services has been outsourcing such processing for some number of
>> financial institutions ... there is folklore about trade-off between
>> having processing to always be under the limit where fine is applied
>> ... versus cost savings on not enough processing for always handling
>> load spikes (offset by the sporadic fines; periodic claims that late
>> processing fines would wipe out any profit on the outsourcing contract)
>> 
> 
> Lynn:
> 
> While this isn't about deposits/checks per se when I used to work at a bank 
> (albeit a long time ago) We had to shutdown the ATM's for one hour every fall 
> and incurred the fines to do so as we could not trust the online application 
> that was using a time stamp for purposes that we couldn't control. I do not 
> recall if it was CICS or a ryo system but the time stamps were critical.
> 
> Ed
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Ed Gould]

On Jun 28, 2012, at 4:00 PM, Anne & Lynn Wheeler wrote:


ibm services has been outsourcing such processing for some number of
financial institutions ... there is folklore about trade-off between
having processing to always be under the limit where fine is applied
... versus cost savings on not enough processing for always handling
load spikes (offset by the sporadic fines; periodic claims that late
processing fines would wipe out any profit on the outsourcing  
contract)




Lynn:

While this isn't about deposits/checks per se when I used to work at  
a bank (albeit a long time ago) We had to shutdown the ATM's for one  
hour every fall and incurred the fines to do so as we could not trust  
the online application that was using a time stamp for purposes that  
we couldn't control. I do not recall if it was CICS or a ryo system  
but the time stamps were critical.


Ed

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Anne & Lynn Wheeler]

david.jurasc...@associates.dhs.gov (Dave Juraschek) writes:
> No.  This is a general accounting rule.  (And actually, the rule is
> that transactions should be processed in the order received by an
> agent of the company/bank.)
> It is a "bank", not a "banking" rule that you allude to.
> I have personally experienced many banks (and subsequently moved my
> holdings to a new bank once I found this out) which apply withdrawls
> first, apply fines if this causes balance issues, and then apply
> deposits.  I suggest that this is actually the "banking" rule used by
> most banks.

ibm services has been outsourcing such processing for some number of
financial institutions ... there is folklore about trade-off between
having processing to always be under the limit where fine is applied
... versus cost savings on not enough processing for always handling
load spikes (offset by the sporadic fines; periodic claims that late
processing fines would wipe out any profit on the outsourcing contract)

-- 
virtualization experience starting Jan1968, online at home since Mar1970

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Kirk Talman]

IBM Mainframe Discussion List  wrote on 
06/28/2012 04:07:46 PM:

> From: Dave Juraschek 
> Mike said:
> >Banking rules say you have to process the day's deposits before the
> >day's withdrawals.
> >Basically, you post the transaction to a transaction file which is
> >applied to the account after the cutoff time.
> 
> No.  This is a general accounting rule.  (And actually, the rule is 
> that transactions should be processed in the order received by an 
> agent of the company/bank.)
> It is a "bank", not a "banking" rule that you allude to.

The rules vary from "bank" to "bank" and from country to country, and in 
some cases the type of account.  In the USA it used to be possible for 
rules to vary from state to state, but I have not seen cases of that 
lately.

There are also rules that vary in a similar way concerning the application 
of fees and the calculation of interest.

The classes I have taken on this have caveats on each software option as 
to where and how it may be used.  Reminds me of standing in a waterfall.

Institutions subject to these rules, including service providers, have 
departments called "compliance" whose job is to make sure rules are 
followed and that advice given is accurate for the situation.

And as Dave indicated in the part of his message not copied, institutions 
use differences in rules to competitive advantage.

-
The information contained in this communication (including any
attachments hereto) is confidential and is intended solely for the
personal and confidential use of the individual or entity to whom
it is addressed. If the reader of this message is not the intended
recipient or an agent responsible for delivering it to the intended
recipient, you are hereby notified that you have received this
communication in error and that any review, dissemination, copying,
or unauthorized use of this information, or the taking of any
action in reliance on the contents of this information is strictly
prohibited. If you have received this communication in error,
please notify us immediately by e-mail, and delete the original
message. Thank you 

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Dave Juraschek]

Mike said:
>Banking rules say you have to process the day's deposits before the
>day's withdrawals.
>Basically, you post the transaction to a transaction file which is
>applied to the account after the cutoff time.

No.  This is a general accounting rule.  (And actually, the rule is that 
transactions should be processed in the order received by an agent of the 
company/bank.)
It is a "bank", not a "banking" rule that you allude to.
I have personally experienced many banks  (and subsequently moved my holdings 
to a new bank once I found this out) which apply withdrawls first, apply fines 
if this causes balance issues, and then apply deposits.  I suggest that this is 
actually the "banking" rule used by most banks.
Worse yet, are the banks which hold deposits for days (most importantly 
including government checks and direct deposits or banking transfers which 
should be instantaneous, IMHO) solely so that they can gain interest on held 
client monies *before* they credit the deposit to their client - despite what 
effect that has on their client.  (I know of several elderly folk who have been 
burned by very big, national banks who practice this regularly on SSI or 
Military payments from the government to them.)  
The entire world wide banking system is corrupt and fully self serving.  Look 
at how Iceland has reacted to this, actions taken and reforms enacted.

It's this client cavalier attitude of the banking industry that looks for cheap 
vs. quality solutions.  From the articles cited thusfar, this debacle is a 
result of that mindset.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Kirk Talman]

well said

not supposed to exist, but ...

especially ugly when the problem is regression testing

IBM Mainframe Discussion List  wrote on 
06/27/2012 05:04:52 PM:

> From: Scott Ford 
> 
> Sounded like production testing..if it works it's productions if notits 
a test
> 
> Scott ford
> www.identityforge.com


-
The information contained in this communication (including any
attachments hereto) is confidential and is intended solely for the
personal and confidential use of the individual or entity to whom
it is addressed. If the reader of this message is not the intended
recipient or an agent responsible for delivering it to the intended
recipient, you are hereby notified that you have received this
communication in error and that any review, dissemination, copying,
or unauthorized use of this information, or the taking of any
action in reliance on the contents of this information is strictly
prohibited. If you have received this communication in error,
please notify us immediately by e-mail, and delete the original
message. Thank you 

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[McKown, John]

> -Original Message-
> From: IBM Mainframe Discussion List 
> [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of David Crayford
> Sent: Thursday, June 28, 2012 8:35 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led 
> to banking meltdown
> 
> The concept of RDz is good. Give newbies an IDE which they're 
> used to. 
> Unfortunately for seasoned mainframer's it just doesn't cut 
> it. It's too 
> slow and click this, right click that etc is just stupid when 
> compared 
> to a command line. It's the same concept as a *nix admin 
> prefering the 
> shell to using a GUI. They get so much more value add from 
> using a shell 
> to a GUI. Editors win every time. Check out www.slickedit.com 
> for your 
> COBOL/HLASM to see what a mature editor can do. Compuware 
> have licensed 
> it for their productivity suite. RDz does not come close.

I'll look at it. Of course, I doubt the company would license it. I did a fast 
look and for a "Named User" (my personal copy) for Linux, the cost is $299.00 . 
Which makes it fairly expensive for me to buy for myself. The plus is that I 
can use it on multiple PCs, so long as only I use it and on one PC at a time. 
Which makes it perfect to be my personal editor, and keep it on a single flash 
drive which I take around with me.

--
John McKown 
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets(r)

9151 Boulevard 26 * N. Richland Hills * TX 76010
(817) 255-3225 phone * 
john.mck...@healthmarkets.com * www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or 
proprietary information. If you are not the intended recipient, please contact 
the sender by reply e-mail and destroy all copies of the original message. 
HealthMarkets(r) is the brand name for products underwritten and issued by the 
insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance 
Company(r), Mid-West National Life Insurance Company of TennesseeSM and The 
MEGA Life and Health Insurance Company.SM

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[David Crayford]

The concept of RDz is good. Give newbies an IDE which they're used to. 
Unfortunately for seasoned mainframer's it just doesn't cut it. It's too 
slow and click this, right click that etc is just stupid when compared 
to a command line. It's the same concept as a *nix admin prefering the 
shell to using a GUI. They get so much more value add from using a shell 
to a GUI. Editors win every time. Check out www.slickedit.com for your 
COBOL/HLASM to see what a mature editor can do. Compuware have licensed 
it for their productivity suite. RDz does not come close.


On 28/06/2012 8:44 PM, McKown, John wrote:

Many thanks for that review. I tend to take the "reports on productivity" from 
"experts" with varying amounts of salt, depending on my experience with the reviewer. 
Some reviewers are given as much salt as anchovies.

My personal desire would be something akin to ISPF available from a z/OS UNIX shell (like 
from telnet or putty, not TSO). I don't know how ISPF is architected. That is, how 
dependant it is on TSO and 3270. I'd expect very dependant. But I would love it if it 
could use a "curses" or X terminal interface, and maybe the TSO dependant parts 
would work properly in an IKJTSOEV environment. However, I doubt that IBM will ever do 
this because they seem to be pushing RDz as the wave of the future. They may be correct 
in that. I have used the basic Eclipse on Linux/Intel and it is a nice IDE. For Java, I 
prefer Netbeans. Probably because I learned it before Eclipse existed. And for simple 
editing, I prefer vim or gvim. But I've also use kate, which is nice for multi-window 
editing.



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[McKown, John]

Many thanks for that review. I tend to take the "reports on productivity" from 
"experts" with varying amounts of salt, depending on my experience with the 
reviewer. Some reviewers are given as much salt as anchovies.

My personal desire would be something akin to ISPF available from a z/OS UNIX 
shell (like from telnet or putty, not TSO). I don't know how ISPF is 
architected. That is, how dependant it is on TSO and 3270. I'd expect very 
dependant. But I would love it if it could use a "curses" or X terminal 
interface, and maybe the TSO dependant parts would work properly in an IKJTSOEV 
environment. However, I doubt that IBM will ever do this because they seem to 
be pushing RDz as the wave of the future. They may be correct in that. I have 
used the basic Eclipse on Linux/Intel and it is a nice IDE. For Java, I prefer 
Netbeans. Probably because I learned it before Eclipse existed. And for simple 
editing, I prefer vim or gvim. But I've also use kate, which is nice for 
multi-window editing.

-- 
John McKown 
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets(r)

9151 Boulevard 26 * N. Richland Hills * TX 76010
(817) 255-3225 phone * 
john.mck...@healthmarkets.com * www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or 
proprietary information. If you are not the intended recipient, please contact 
the sender by reply e-mail and destroy all copies of the original message. 
HealthMarkets(r) is the brand name for products underwritten and issued by the 
insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance 
Company(r), Mid-West National Life Insurance Company of TennesseeSM and The 
MEGA Life and Health Insurance Company.SM

> -Original Message-
> From: IBM Mainframe Discussion List 
> [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of David Crayford
> Sent: Thursday, June 28, 2012 1:12 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led 
> to banking meltdown
> 
> On 27/06/2012 11:23 PM, McKown, John wrote:
> > I somewhat agree (as difficult as that is for me). In 
> today's world, it is expected that a worker will just walk in 
> off the street and have the "intuitive" knowledge of how to 
> use computers. As much as I dislike it personally, z/OS 
> really needs the "new look" interface to present to the end 
> users, and even programmers. I wonder if anybody has done a 
> study of productivity between "old style" development using 
> ISPF and edit-compile-test versus using the RD/z Eclipse 
> based software.
> 
> I can tell you from experience (I use RDz) that other than the actual 
> smart context assisting editors the tooling in RDz is 
> generally poorly 
> implemented and less productive than using the ISPF equivalents.
> Of course, I'm not talking about all the application web 
> enabling stuff. 
> OTTOMH, FileManager, FaultAnalyzer, 
> ApplicationPerformanceAnalyzer all 
> have plug-ins that are inferior to
> the 3270 apps. It's a shame because they could have been much 
> better.  
> Open a big dump in RDz from the spool system explorer, wait 
> 15 minutes, 
> Eclipse chokes out an out-of-memory stack trace.
> The mouse has to go down in history as possibly the worst 
> invention for 
> hindering productivity. When I have to right-click and select from a 
> menu as apposed to tabbing and hitting a key then I generally
> despise the UI and don't use it.
> 
> I also gave up on compiling in RDz. It's so much easier just 
> to fire up 
> a shell or run JCL in a separate window. Even easier in 
> Slickedit where 
> I can just use Putty and re-direct the output to my build window.
> It's much easier to edit a Makefile than use a stupid dialog 
> which has 
> unreasonable constraints.
> 
> > --
> > John McKown
> > Systems Engineer IV
> > IT
> >
> > Administrative Services Group
> >
> > HealthMarkets(r)
> >
> > 9151 Boulevard 26 . N. Richland Hills . TX 76010
> > (817) 255-3225 phone .
> > john.mck...@healthmarkets.com . www.HealthMarkets.com
> >
> > Confidentiality Notice: This e-mail message may contain 
> confidential or proprietary information. If you are not the 
> intended recipient, please contact the sender by reply e-mail 
> and destroy all copies of the original message. 
> HealthMarkets(r) is the brand name for products underwritten 
> and issued by the insurance subsidiaries of HealthMarkets, 
> Inc. -The Chesapeake Life Insurance Company(r), Mid-West 
> National Life Insurance Company of TennesseeSM and The MEGA 
> Life and Health Insurance Company.SM
> >
> >   
> >

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[David Crayford]

On 27/06/2012 11:23 PM, McKown, John wrote:

I somewhat agree (as difficult as that is for me). In today's world, it is expected that a worker will just 
walk in off the street and have the "intuitive" knowledge of how to use computers. As much as I 
dislike it personally, z/OS really needs the "new look" interface to present to the end users, and 
even programmers. I wonder if anybody has done a study of productivity between "old style" 
development using ISPF and edit-compile-test versus using the RD/z Eclipse based software.


I can tell you from experience (I use RDz) that other than the actual 
smart context assisting editors the tooling in RDz is generally poorly 
implemented and less productive than using the ISPF equivalents.
Of course, I'm not talking about all the application web enabling stuff. 
OTTOMH, FileManager, FaultAnalyzer, ApplicationPerformanceAnalyzer all 
have plug-ins that are inferior to
the 3270 apps. It's a shame because they could have been much better.  
Open a big dump in RDz from the spool system explorer, wait 15 minutes, 
Eclipse chokes out an out-of-memory stack trace.
The mouse has to go down in history as possibly the worst invention for 
hindering productivity. When I have to right-click and select from a 
menu as apposed to tabbing and hitting a key then I generally

despise the UI and don't use it.

I also gave up on compiling in RDz. It's so much easier just to fire up 
a shell or run JCL in a separate window. Even easier in Slickedit where 
I can just use Putty and re-direct the output to my build window.
It's much easier to edit a Makefile than use a stupid dialog which has 
unreasonable constraints.



--
John McKown
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets®

9151 Boulevard 26 . N. Richland Hills . TX 76010
(817) 255-3225 phone .
john.mck...@healthmarkets.com . www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or 
proprietary information. If you are not the intended recipient, please contact 
the sender by reply e-mail and destroy all copies of the original message. 
HealthMarkets® is the brand name for products underwritten and issued by the 
insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance 
Company®, Mid-West National Life Insurance Company of TennesseeSM and The MEGA 
Life and Health Insurance Company.SM

  


-Original Message-
From: IBM Mainframe Discussion List
[mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Knutson, Sam
Sent: Wednesday, June 27, 2012 8:49 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: 'Inexperienced' RBS tech operative's blunder led
to banking meltdown

To someone who is inexperienced on platform I imagine
managing a CA-7 upgrade is like having the world's smartest
dog but it only responds to commands in Latin.
Miscommunication is likely to be a source of dissatisfaction.

The average person <30 is not stupid because they haven't
been editing command decks and JCL for 25 years or lazy but
software on our platform tends to be very particular and
sometimes unforgiving of mistakes i.e. failure to specify
QUEUE=NOFORMAT or some non-obvious phrase as part of START
command.   Since business is no longer content to bring in
newbies and let them spend 5-10 years to reach Journeyman
status under the guidance of more senior folks software is
going to have to get smarter and more forgiving,
documentation is going to have to be written with less
assumptions about the expertise of the "systems programmer"
and zNextGen (generic) is going to operate "our" mainframes
weather we think they are ready or not.  Some of those next
generation folks are not going to be based in the home office
anymore either whether we like that or not.

The Register is fun reading sometimes but they rarely have
enough detail to get real insight into what actually
happened.  So really anything we say here is just idle
speculation.   Feels like Friday so that's my .02  worth of
idle speculation :-)

 Best Regards,

 Sam Knutson, GEICO
 System z Team Leader
 mailto:sknut...@geico.com
 (office)  301.986.3574
 (cell) 301.996.1318

"Think big, act bold, start simple, grow fast..."



-Original Message-
From: IBM Mainframe Discussion List
[mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Tom Ambros
Sent: Wednesday, June 27, 2012 8:25 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: 'Inexperienced' RBS tech operative's blunder led
to banking meltdown

"Complicated legacy mainframe system"?  It's CA-7!  If they
think that's complicated they probably can't work a Mr. Coffee.

Thomas Ambros
Operating Systems and Connectivity Engineering
518-436-6433





From:   Ed Gould 
To: IBM-MAIN@LISTSERV.UA.EDU
D

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Scott Ford]

Jerry,

I went to a DR test, where before hand myself and another old hand tried to 
tell some younger staff members make sure they inventory their tapes..well the 
didn't and forgot 14 of them

Scott ford
www.identityforge.com

On Jun 27, 2012, at 1:11 PM, Jerry Whitteridge  
wrote:

> I also use the residual time from our DR tests for "Destructive testing". 
> It's a great way to test procedures at no risk and do some training that 
> normally can't be done due to risk.
> 
> Jerry Whitteridge
> Lead Systems Programmer
> Safeway Inc.
> 925 951 4184
> 
> If you feel in control
> you just aren't going fast enough.
> 
> 
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On 
> Behalf Of Tom Ambros
> Sent: Wednesday, June 27, 2012 10:07 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking 
> meltdown
> 
> We just happened to upgrade our scheduler earlier this year and the tech 
> doing it took advantage of a DR test to practice the production upgrade 
> and backout, he had done the backout in the sandbox but wanted to see how 
> long production would take.  He happens to be the tech with the least 
> tenure as a tech but with many years of operations and batch support 
> experience... sort of how a lot of people used to end up in tech, I guess. 
> 
> 
> Thomas Ambros
> Operating Systems and Connectivity Engineering
> 518-436-6433
> 
> 
> 
> 
> 
> From:   Jerry Whitteridge 
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date:   06/27/2012 12:48
> Subject:Re: 'Inexperienced' RBS tech operative's blunder led to 
> banking meltdown
> Sent by:IBM Mainframe Discussion List 
> 
> 
> 
> One relates to a conditioning PTF required on the back level system to be 
> able to backout an upgrade from a higher using a backup (DMPQ/MOVQ) 
> process native to CA-7. As I read it they attempted to backout using the 
> standard documented process but the downlevel system couldn't use the data 
> formatted for the uplevel system and queue corruption might have occurred.
> 
> It's  shutting the door after the event - but also points to a code not a 
> process problem.
> 
> Jerry Whitteridge
> Lead Systems Programmer
> Safeway Inc.
> 925 951 4184
> 
> If you feel in control
> you just aren't going fast enough.
> 
> 
> -----Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On 
> Behalf Of Bill Ashton
> Sent: Wednesday, June 27, 2012 9:39 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking 
> meltdown
> 
> Without having seen the HIPERs, it could be that they are trying to keep
> others from blowing off their feet if they have staff who are
> "operationally challenged."
> B
> 
> On Wed, Jun 27, 2012 at 12:34 PM, Jerry Whitteridge <
> jerry.whitteri...@safeway.com> wrote:
> 
>> I've just seen 3 HIPER PTF's from CA related to CA-7 that also fit the
>> reported information on the RBS outage. I'd not be too hasty in blaming
>> staff at this point.
>> 
>> Jerry Whitteridge
>> Lead Systems Programmer
>> Safeway Inc.
>> 925 951 4184
>> 
>> If you feel in control
>> you just aren't going fast enough.
>> 
>> 
>> -Original Message-
>> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
>> Behalf Of Knutson, Sam
>> Sent: Wednesday, June 27, 2012 6:49 AM
>> To: IBM-MAIN@LISTSERV.UA.EDU
>> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking
>> meltdown
>> 
>> To someone who is inexperienced on platform I imagine managing a CA-7
>> upgrade is like having the world's smartest dog but it only responds to
>> commands in Latin.  Miscommunication is likely to be a source of
>> dissatisfaction.
>> 
>> The average person <30 is not stupid because they haven't been editing
>> command decks and JCL for 25 years or lazy but software on our platform
>> tends to be very particular and sometimes unforgiving of mistakes i.e.
>> failure to specify QUEUE=NOFORMAT or some non-obvious phrase as part of
>> START command.   Since business is no longer content to bring in newbies
>> and let them spend 5-10 years to reach Journeyman status under the 
> guidance
>> of more senior folks software is going to have to get smarter and more
>> forgiving, documentation is going to have to be written with less
>> assumption

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Scott Ford]

Phil,

Had a boss on VM man yrs ago, said learn the native VM and CMS commands first 
before you considering writing execs or clists. That was 20+ yrs ago, did that 
with VM , CMS, VTAM, TCPIP, etc ..if you understand how things work the GUIs 
are to me pretty but also makes people not think my $0.02 worth

Scott ford
www.identityforge.com

On Jun 27, 2012, at 11:43 AM, Phil Smith  wrote:

> John McKown wrote:
>> I somewhat agree (as difficult as that is for me). In today's world, it is 
>> expected that a worker will just walk in off the street and have the 
>> "intuitive" knowledge of how to use computers. As much as I dislike it 
>> personally, z/OS really needs the "new look" interface to present to the end 
>> users, and even programmers. I wonder if anybody has done a study of 
>> productivity between "old style" development using ISPF and 
>> edit-compile-test versus using the RD/z Eclipse based software.
> 
> Of course they have...many times. And they all show that IDEs are more 
> productive, FVSO "more productive": programs get done faster. Not necessarily 
> better, but faster. And it's a positive feedback loop: once the IDE is all 
> anyone has used, old-school programming becomes unthinkable.
> 
> And that's how we get the instability that is Windows. Not even necessarily 
> the Microsoft end of it: a near-infinite number of vendors with an infinite 
> number of monkeys banging away at their IDEs, producing products that sort of 
> work, but perhaps destabilize the underlying OS (or Office, or some other 
> product) may be the real cause.
> 
> Of course this is an oversimplification, but when you make programming so 
> simple, even a kid can do it, you get programs written by kids.
> 
> Even open source, for all its benefits, isn't a real solution: look at the 
> plugin-container hack Firefox has resorted to. Is that strictly because Flash 
> is so unstable? Chrome doesn't seem to have any significant Flash problems 
> (or maybe I just use Firefox more). I'm on the Beta channel of Firefox right 
> now because the production version was hanging my Windows 7 machines (maybe 
> that's been fixed by now, but it *was* true). Worse, I had no real way to 
> debug it.
> 
> OK, so I sound like a cranky old fart. If the shoe fits...
> --
> ...phsiii
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Henri zdevops Kuiper]

Maybe they need a free 2H zdevops session ;)

Sent from my wireless iPhone

On 27 jun. 2012, at 22:58, Scott Ford  wrote:

> After reading the article a lot of issues...technical and 
> non-technicalseems to be a epidemic of ppl who don't want to read 
> ..seeing a lot of it lately. Plus good experienced people are hard to find.
>
> Fwiw
>
> Scott ford
> www.identityforge.com
>
> On Jun 27, 2012, at 12:39 PM, Bill Ashton  wrote:
>
>> Without having seen the HIPERs, it could be that they are trying to keep
>> others from blowing off their feet if they have staff who are
>> "operationally challenged."
>> B
>>
>> On Wed, Jun 27, 2012 at 12:34 PM, Jerry Whitteridge <
>> jerry.whitteri...@safeway.com> wrote:
>>
>>> I've just seen 3 HIPER PTF's from CA related to CA-7 that also fit the
>>> reported information on the RBS outage. I'd not be too hasty in blaming
>>> staff at this point.
>>>
>>> Jerry Whitteridge
>>> Lead Systems Programmer
>>> Safeway Inc.
>>> 925 951 4184
>>>
>>> If you feel in control
>>> you just aren't going fast enough.
>>>
>>>
>>> -Original Message-----
>>> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
>>> Behalf Of Knutson, Sam
>>> Sent: Wednesday, June 27, 2012 6:49 AM
>>> To: IBM-MAIN@LISTSERV.UA.EDU
>>> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking
>>> meltdown
>>>
>>> To someone who is inexperienced on platform I imagine managing a CA-7
>>> upgrade is like having the world's smartest dog but it only responds to
>>> commands in Latin.  Miscommunication is likely to be a source of
>>> dissatisfaction.
>>>
>>> The average person <30 is not stupid because they haven't been editing
>>> command decks and JCL for 25 years or lazy but software on our platform
>>> tends to be very particular and sometimes unforgiving of mistakes i.e.
>>> failure to specify QUEUE=NOFORMAT or some non-obvious phrase as part of
>>> START command.   Since business is no longer content to bring in newbies
>>> and let them spend 5-10 years to reach Journeyman status under the guidance
>>> of more senior folks software is going to have to get smarter and more
>>> forgiving, documentation is going to have to be written with less
>>> assumptions about the expertise of the "systems programmer" and zNextGen
>>> (generic) is going to operate "our" mainframes weather we think they are
>>> ready or not.  Some of those next generation folks are not going to be
>>> based in the home office anymore either whether we like that or not.
>>>
>>> The Register is fun reading sometimes but they rarely have enough detail
>>> to get real insight into what actually happened.  So really anything we say
>>> here is just idle speculation.   Feels like Friday so that's my .02  worth
>>> of idle speculation :-)
>>>
>>>   Best Regards,
>>>
>>>   Sam Knutson, GEICO
>>>   System z Team Leader
>>>   mailto:sknut...@geico.com
>>>   (office)  301.986.3574
>>>   (cell) 301.996.1318
>>>
>>> "Think big, act bold, start simple, grow fast..."
>>>
>>>
>>> -Original Message-
>>> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
>>> Behalf Of Tom Ambros
>>> Sent: Wednesday, June 27, 2012 8:25 AM
>>> To: IBM-MAIN@LISTSERV.UA.EDU
>>> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking
>>> meltdown
>>>
>>> "Complicated legacy mainframe system"?  It's CA-7!  If they think that's
>>> complicated they probably can't work a Mr. Coffee.
>>>
>>> Thomas Ambros
>>> Operating Systems and Connectivity Engineering
>>> 518-436-6433
>>>
>>>
>>>
>>>
>>>
>>> From:   Ed Gould 
>>> To: IBM-MAIN@LISTSERV.UA.EDU
>>> Date:   06/27/2012 00:44
>>> Subject:'Inexperienced' RBS tech operative's blunder led to
>>> banking meltdown
>>> Sent by:IBM Mainframe Discussion List 
>>>
>>>
>>>
>>>
>>> http://www.theregister.co.uk/2012/06/26/rbs_natwest_ca_technologies_outsourcing/
>>>
>>>
>

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Scott Ford]

Ed, 

Sounded like production testing..if it works it's productions if not its a test

Scott ford
www.identityforge.com

On Jun 27, 2012, at 3:47 PM, Ed Gould  wrote:

> Could it be instead that CA does not follow SMPE conventions (sup,pre etc) ?
> 
> Ed
> 
> ps: I would love to see it that was the real reason. Only then CA would be 
> embarrassed (only after 10+ years)!
> 
> 
> On Jun 27, 2012, at 11:46 AM, Jerry Whitteridge wrote:
> 
>> One relates to a conditioning PTF required on the back level system to be 
>> able to backout an upgrade from a higher using a backup (DMPQ/MOVQ) process 
>> native to CA-7. As I read it they attempted to backout using the standard 
>> documented process but the downlevel system couldn't use the data formatted 
>> for the uplevel system and queue corruption might have occurred.
>> 
>> It's  shutting the door after the event - but also points to a code not a 
>> process problem.
>> 
>> Jerry Whitteridge
>> Lead Systems Programmer
>> Safeway Inc.
>> 925 951 4184
>> 
>> If you feel in control
>> you just aren't going fast enough.
>> 
>> 
>> -Original Message-
>> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On 
>> Behalf Of Bill Ashton
>> Sent: Wednesday, June 27, 2012 9:39 AM
>> To: IBM-MAIN@LISTSERV.UA.EDU
>> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking 
>> meltdown
>> 
>> Without having seen the HIPERs, it could be that they are trying to keep
>> others from blowing off their feet if they have staff who are
>> "operationally challenged."
>> B
>> 
>> On Wed, Jun 27, 2012 at 12:34 PM, Jerry Whitteridge <
>> jerry.whitteri...@safeway.com> wrote:
>> 
>>> I've just seen 3 HIPER PTF's from CA related to CA-7 that also fit the
>>> reported information on the RBS outage. I'd not be too hasty in blaming
>>> staff at this point.
>>> 
>>> Jerry Whitteridge
>>> Lead Systems Programmer
>>> Safeway Inc.
>>> 925 951 4184
>>> 
>>> If you feel in control
>>> you just aren't going fast enough.
>>> 
>>> 
>>> -Original Message-
>>> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
>>> Behalf Of Knutson, Sam
>>> Sent: Wednesday, June 27, 2012 6:49 AM
>>> To: IBM-MAIN@LISTSERV.UA.EDU
>>> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking
>>> meltdown
>>> 
>>> To someone who is inexperienced on platform I imagine managing a CA-7
>>> upgrade is like having the world's smartest dog but it only responds to
>>> commands in Latin.  Miscommunication is likely to be a source of
>>> dissatisfaction.
>>> 
>>> The average person <30 is not stupid because they haven't been editing
>>> command decks and JCL for 25 years or lazy but software on our platform
>>> tends to be very particular and sometimes unforgiving of mistakes i.e.
>>> failure to specify QUEUE=NOFORMAT or some non-obvious phrase as part of
>>> START command.   Since business is no longer content to bring in newbies
>>> and let them spend 5-10 years to reach Journeyman status under the guidance
>>> of more senior folks software is going to have to get smarter and more
>>> forgiving, documentation is going to have to be written with less
>>> assumptions about the expertise of the "systems programmer" and zNextGen
>>> (generic) is going to operate "our" mainframes weather we think they are
>>> ready or not.  Some of those next generation folks are not going to be
>>> based in the home office anymore either whether we like that or not.
>>> 
>>> The Register is fun reading sometimes but they rarely have enough detail
>>> to get real insight into what actually happened.  So really anything we say
>>> here is just idle speculation.   Feels like Friday so that's my .02  worth
>>> of idle speculation :-)
>>> 
>>>Best Regards,
>>> 
>>>Sam Knutson, GEICO
>>>System z Team Leader
>>>mailto:sknut...@geico.com
>>>(office)  301.986.3574
>>>(cell) 301.996.1318
>>> 
>>> "Think big, act bold, start simple, grow fast..."
>>> 
>>> 
>>> -Original Message-
>>> From: IBM Mainframe Discussion 

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Scott Ford]

After reading the article a lot of issues...technical and 
non-technicalseems to be a epidemic of ppl who don't want to read ..seeing 
a lot of it lately. Plus good experienced people are hard to find. 

Fwiw

Scott ford
www.identityforge.com

On Jun 27, 2012, at 12:39 PM, Bill Ashton  wrote:

> Without having seen the HIPERs, it could be that they are trying to keep
> others from blowing off their feet if they have staff who are
> "operationally challenged."
> B
> 
> On Wed, Jun 27, 2012 at 12:34 PM, Jerry Whitteridge <
> jerry.whitteri...@safeway.com> wrote:
> 
>> I've just seen 3 HIPER PTF's from CA related to CA-7 that also fit the
>> reported information on the RBS outage. I'd not be too hasty in blaming
>> staff at this point.
>> 
>> Jerry Whitteridge
>> Lead Systems Programmer
>> Safeway Inc.
>> 925 951 4184
>> 
>> If you feel in control
>> you just aren't going fast enough.
>> 
>> 
>> -Original Message-
>> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
>> Behalf Of Knutson, Sam
>> Sent: Wednesday, June 27, 2012 6:49 AM
>> To: IBM-MAIN@LISTSERV.UA.EDU
>> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking
>> meltdown
>> 
>> To someone who is inexperienced on platform I imagine managing a CA-7
>> upgrade is like having the world's smartest dog but it only responds to
>> commands in Latin.  Miscommunication is likely to be a source of
>> dissatisfaction.
>> 
>> The average person <30 is not stupid because they haven't been editing
>> command decks and JCL for 25 years or lazy but software on our platform
>> tends to be very particular and sometimes unforgiving of mistakes i.e.
>> failure to specify QUEUE=NOFORMAT or some non-obvious phrase as part of
>> START command.   Since business is no longer content to bring in newbies
>> and let them spend 5-10 years to reach Journeyman status under the guidance
>> of more senior folks software is going to have to get smarter and more
>> forgiving, documentation is going to have to be written with less
>> assumptions about the expertise of the "systems programmer" and zNextGen
>> (generic) is going to operate "our" mainframes weather we think they are
>> ready or not.  Some of those next generation folks are not going to be
>> based in the home office anymore either whether we like that or not.
>> 
>> The Register is fun reading sometimes but they rarely have enough detail
>> to get real insight into what actually happened.  So really anything we say
>> here is just idle speculation.   Feels like Friday so that's my .02  worth
>> of idle speculation :-)
>> 
>>Best Regards,
>> 
>>Sam Knutson, GEICO
>>System z Team Leader
>>        mailto:sknut...@geico.com
>>    (office)  301.986.3574
>>(cell) 301.996.1318
>> 
>> "Think big, act bold, start simple, grow fast..."
>> 
>> 
>> -Original Message-
>> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
>> Behalf Of Tom Ambros
>> Sent: Wednesday, June 27, 2012 8:25 AM
>> To: IBM-MAIN@LISTSERV.UA.EDU
>> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking
>> meltdown
>> 
>> "Complicated legacy mainframe system"?  It's CA-7!  If they think that's
>> complicated they probably can't work a Mr. Coffee.
>> 
>> Thomas Ambros
>> Operating Systems and Connectivity Engineering
>> 518-436-6433
>> 
>> 
>> 
>> 
>> 
>> From:   Ed Gould 
>> To: IBM-MAIN@LISTSERV.UA.EDU
>> Date:   06/27/2012 00:44
>> Subject:'Inexperienced' RBS tech operative's blunder led to
>> banking meltdown
>> Sent by:IBM Mainframe Discussion List 
>> 
>> 
>> 
>> 
>> http://www.theregister.co.uk/2012/06/26/rbs_natwest_ca_technologies_outsourcing/
>> 
>> 
>> 
>> 
>> This email/fax message is for the sole use of the intended
>> recipient(s) and may contain confidential and privileged information.
>> Any unauthorized review, use, disclosure or distribution of this
>> email/fax is prohibited. If you are not the intended recipient, please
>> destroy all paper and electronic copies of the original message.
>> 
>> --
>> For IBM-MAIN 

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Phil Smith]

Frank Swarbrick wrote:
>I thought all "systems programmers" were perfect?

Well, sum of us are.




















(Yes, that was deliberate!)

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Ed Gould]

Could it be instead that CA does not follow SMPE conventions (sup,pre  
etc) ?


Ed

ps: I would love to see it that was the real reason. Only then CA  
would be embarrassed (only after 10+ years)!



On Jun 27, 2012, at 11:46 AM, Jerry Whitteridge wrote:

One relates to a conditioning PTF required on the back level system  
to be able to backout an upgrade from a higher using a backup (DMPQ/ 
MOVQ) process native to CA-7. As I read it they attempted to  
backout using the standard documented process but the downlevel  
system couldn't use the data formatted for the uplevel system and  
queue corruption might have occurred.


It's  shutting the door after the event - but also points to a code  
not a process problem.


Jerry Whitteridge
Lead Systems Programmer
Safeway Inc.
925 951 4184

If you feel in control
you just aren't going fast enough.


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM- 
m...@listserv.ua.edu] On Behalf Of Bill Ashton

Sent: Wednesday, June 27, 2012 9:39 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: 'Inexperienced' RBS tech operative's blunder led to  
banking meltdown


Without having seen the HIPERs, it could be that they are trying to  
keep

others from blowing off their feet if they have staff who are
"operationally challenged."
B

On Wed, Jun 27, 2012 at 12:34 PM, Jerry Whitteridge <
jerry.whitteri...@safeway.com> wrote:

I've just seen 3 HIPER PTF's from CA related to CA-7 that also fit  
the
reported information on the RBS outage. I'd not be too hasty in  
blaming

staff at this point.

Jerry Whitteridge
Lead Systems Programmer
Safeway Inc.
925 951 4184

If you feel in control
you just aren't going fast enough.


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM- 
m...@listserv.ua.edu] On

Behalf Of Knutson, Sam
Sent: Wednesday, June 27, 2012 6:49 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: 'Inexperienced' RBS tech operative's blunder led to  
banking

meltdown

To someone who is inexperienced on platform I imagine managing a CA-7
upgrade is like having the world's smartest dog but it only  
responds to

commands in Latin.  Miscommunication is likely to be a source of
dissatisfaction.

The average person <30 is not stupid because they haven't been  
editing
command decks and JCL for 25 years or lazy but software on our  
platform
tends to be very particular and sometimes unforgiving of mistakes  
i.e.
failure to specify QUEUE=NOFORMAT or some non-obvious phrase as  
part of
START command.   Since business is no longer content to bring in  
newbies
and let them spend 5-10 years to reach Journeyman status under the  
guidance
of more senior folks software is going to have to get smarter and  
more

forgiving, documentation is going to have to be written with less
assumptions about the expertise of the "systems programmer" and  
zNextGen
(generic) is going to operate "our" mainframes weather we think  
they are
ready or not.  Some of those next generation folks are not going  
to be

based in the home office anymore either whether we like that or not.

The Register is fun reading sometimes but they rarely have enough  
detail
to get real insight into what actually happened.  So really  
anything we say
here is just idle speculation.   Feels like Friday so that's my . 
02  worth

of idle speculation :-)

Best Regards,

Sam Knutson, GEICO
System z Team Leader
mailto:sknut...@geico.com
(office)  301.986.3574
(cell) 301.996.1318

"Think big, act bold, start simple, grow fast..."


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM- 
m...@listserv.ua.edu] On

Behalf Of Tom Ambros
Sent: Wednesday, June 27, 2012 8:25 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: 'Inexperienced' RBS tech operative's blunder led to  
banking

meltdown

"Complicated legacy mainframe system"?  It's CA-7!  If they think  
that's

complicated they probably can't work a Mr. Coffee.

Thomas Ambros
Operating Systems and Connectivity Engineering
518-436-6433





From:   Ed Gould 
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   06/27/2012 00:44
Subject:'Inexperienced' RBS tech operative's blunder led to
banking meltdown
Sent by:IBM Mainframe Discussion List m...@listserv.ua.edu>





http://www.theregister.co.uk/2012/06/26/ 
rbs_natwest_ca_technologies_outsourcing/





This email/fax message is for the sole use of the intended
recipient(s) and may contain confidential and privileged information.
Any unauthorized review, use, disclosure or distribution of this
email/fax is prohibited. If you are not the intended recipient,  
please

destroy all paper and electronic copies of the original message.

--

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Mike Schwab]

On Wed, Jun 27, 2012 at 10:52 AM, Phil Smith  wrote:
> Bill Fairchild wrote:
>>I understand why all accumulated transactions have to be processed in the 
>>correct sequence if they are batched up daily, but why is an individual 
>>transaction not instantaneously and permanently processed with the results 
>>being committed right then?  Why is it necessary to run huge batches of 
>>transactions at night?   I am guessing that committing the transaction 
>>instantly would take too long for the average customer's patience when 
>>standing at an outdoors ATM in the rain.  Are there any other reasons?  When 
>>I buy an airplane ticket online, I get my whole transaction complete in one 
>>sitting and do not have to wait until the next day to receive confirmation of 
>>my reservation, but it always takes several minutes to do it online.
>
> Besides/in addition to Mike Schwab's point, credit and signature debit 
> transactions aren't processed in real-time: an authorization is processed, 
> but the actual charge/debit doesn't go through until settlement.
>
> Come to session 11409: The Payments Ecosystem: Security Challenges in 
> the 21st 
> Century at 
> SHARE in Anaheim, Monday at 3PM, to learn more about how this
> --
> ...phsiii

Depositing checks by using a smart phone camera to electronic process
checks and image recognition by the bank is gaining traction.
Clearing the check image through the paying bank still takes time.
http://www.bankrate.com/financing/banking/more-banks-ok-smartphone-deposits/

Now, using a smart phone app to schedule an ATM withdraw instead of
using ATM cards and PINs hopes to end ATM card skimming and recording
the user entering the PIN.
http://www.bankrate.com/financing/banking/atm-withdrawals-without-a-card/?ec_id=m1078093

-- 
Mike A Schwab, Springfield IL USA
Where do Forest Rangers go to get away from it all?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Ken Porowski]

FSVO "perfect"


--
This email message and any accompanying materials may contain proprietary, 
privileged and confidential information of CIT Group Inc. or its subsidiaries 
or affiliates (collectively, "CIT"), and are intended solely for 
the recipient(s) named above. If you are not the intended recipient of this 
communication, any use, disclosure, printing, copying or distribution, or 
reliance on the contents, of this communication is strictly prohibited. CIT 
disclaims any liability for the review, retransmission, dissemination or other 
use of, or the taking of any action in reliance upon, this communication by 
persons other than the intended recipient(s). If you have received this 
communication in error, please reply to the sender advising of the error in 
transmission, and immediately delete and destroy the communication and any 
accompanying materials. To the extent permitted by applicable law, CIT and 
others may inspect, review, monitor, analyze, copy, record and retain any 
communications sent from or received at this email address.
--

Frank Swarbrick

I thought all "systems programmers" were perfect?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Ed Finnell]

We used to subscribe to weekly world news, before times got tough. Then  
there was the AA fiasco back in 80's where they managed to reply U to scratch  
VTOC purge-1100 times!
 
 
In a message dated 6/27/2012 9:45:49 A.M. Central Daylight Time,  
m...@mzelden.com writes:

I told  him to think
of it as the "National Enquirer of IT".   :-)



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Frank Swarbrick]

I thought all "systems programmers" were perfect?




>
> From: Ed Gould 
>To: IBM-MAIN@LISTSERV.UA.EDU 
>Sent: Tuesday, June 26, 2012 10:37 PM
>Subject: 'Inexperienced' RBS tech operative's blunder led to banking meltdown
> 
>http://www.theregister.co.uk/2012/06/26/rbs_natwest_ca_technologies_outsourcing/
>
>
>--
>For IBM-MAIN subscribe / signoff / archive access instructions,
>send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>
>
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[McKown, John]

Maybe. But all we have here is TSO/ISPF/SDSF along with some CompuWare products 
(Xpeditor and AbendAid mainly) to help us with development and testing. If "we" 
could get away with it, we'd likely get rid of ISPF/SDSF (not CompuWare, it is 
off maintenance and is perm licensed) to reduce costs. We look only at cost. 
Not value. Not productivity. Cost only. That may loosen up, depending of what 
SCOTUS says about Obamacare.

-- 
John McKown 
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets(r)

9151 Boulevard 26 * N. Richland Hills * TX 76010
(817) 255-3225 phone * 
john.mck...@healthmarkets.com * www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or 
proprietary information. If you are not the intended recipient, please contact 
the sender by reply e-mail and destroy all copies of the original message. 
HealthMarkets(r) is the brand name for products underwritten and issued by the 
insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance 
Company(r), Mid-West National Life Insurance Company of TennesseeSM and The 
MEGA Life and Health Insurance Company.SM

> -Original Message-
> From: IBM Mainframe Discussion List 
> [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Dave Salt
> Sent: Wednesday, June 27, 2012 11:16 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led 
> to banking meltdown
> 
> > From: john.mck...@healthmarkets.com
> >I wonder if anybody has 
> done a study of productivity between "old style" development 
> using ISPF 
> and edit-compile-test versus using the RD/z Eclipse based software.
> > 
> 
> Well if they did I hope they at least used SimpList or IPT 
> instead of vanilla ISPF, otherwise they'd be painting the 
> mainframe in a very unproductive and unfair light.
>   
> Dave Salt
> 
> SimpList(tm) - try it; you'll get it! 
> 
> http://www.mackinney.com/products/program-development/simplist.html  
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> 
> 
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Jerry Whitteridge]

I also use the residual time from our DR tests for "Destructive testing". It's 
a great way to test procedures at no risk and do some training that normally 
can't be done due to risk.

Jerry Whitteridge
Lead Systems Programmer
Safeway Inc.
925 951 4184

If you feel in control
you just aren't going fast enough.


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Tom Ambros
Sent: Wednesday, June 27, 2012 10:07 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking 
meltdown

We just happened to upgrade our scheduler earlier this year and the tech 
doing it took advantage of a DR test to practice the production upgrade 
and backout, he had done the backout in the sandbox but wanted to see how 
long production would take.  He happens to be the tech with the least 
tenure as a tech but with many years of operations and batch support 
experience... sort of how a lot of people used to end up in tech, I guess. 
 

Thomas Ambros
Operating Systems and Connectivity Engineering
518-436-6433





From:   Jerry Whitteridge 
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   06/27/2012 12:48
Subject:Re: 'Inexperienced' RBS tech operative's blunder led to 
banking meltdown
Sent by:IBM Mainframe Discussion List 



One relates to a conditioning PTF required on the back level system to be 
able to backout an upgrade from a higher using a backup (DMPQ/MOVQ) 
process native to CA-7. As I read it they attempted to backout using the 
standard documented process but the downlevel system couldn't use the data 
formatted for the uplevel system and queue corruption might have occurred.

It's  shutting the door after the event - but also points to a code not a 
process problem.

Jerry Whitteridge
Lead Systems Programmer
Safeway Inc.
925 951 4184

If you feel in control
you just aren't going fast enough.


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On 
Behalf Of Bill Ashton
Sent: Wednesday, June 27, 2012 9:39 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking 
meltdown

Without having seen the HIPERs, it could be that they are trying to keep
others from blowing off their feet if they have staff who are
"operationally challenged."
B

On Wed, Jun 27, 2012 at 12:34 PM, Jerry Whitteridge <
jerry.whitteri...@safeway.com> wrote:

> I've just seen 3 HIPER PTF's from CA related to CA-7 that also fit the
> reported information on the RBS outage. I'd not be too hasty in blaming
> staff at this point.
>
> Jerry Whitteridge
> Lead Systems Programmer
> Safeway Inc.
> 925 951 4184
>
> If you feel in control
> you just aren't going fast enough.
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Knutson, Sam
> Sent: Wednesday, June 27, 2012 6:49 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking
> meltdown
>
> To someone who is inexperienced on platform I imagine managing a CA-7
> upgrade is like having the world's smartest dog but it only responds to
> commands in Latin.  Miscommunication is likely to be a source of
> dissatisfaction.
>
> The average person <30 is not stupid because they haven't been editing
> command decks and JCL for 25 years or lazy but software on our platform
> tends to be very particular and sometimes unforgiving of mistakes i.e.
> failure to specify QUEUE=NOFORMAT or some non-obvious phrase as part of
> START command.   Since business is no longer content to bring in newbies
> and let them spend 5-10 years to reach Journeyman status under the 
guidance
> of more senior folks software is going to have to get smarter and more
> forgiving, documentation is going to have to be written with less
> assumptions about the expertise of the "systems programmer" and zNextGen
> (generic) is going to operate "our" mainframes weather we think they are
> ready or not.  Some of those next generation folks are not going to be
> based in the home office anymore either whether we like that or not.
>
> The Register is fun reading sometimes but they rarely have enough detail
> to get real insight into what actually happened.  So really anything we 
say
> here is just idle speculation.   Feels like Friday so that's my .02 
worth
> of idle speculation :-)
>
> Best Regards,
>
> Sam Knutson, GEICO
> System z Team Leader
> mailto:sknut...@geico.com
> (office)  301.986.3574
>     (cell) 301.996.1

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Tom Ambros]

We just happened to upgrade our scheduler earlier this year and the tech 
doing it took advantage of a DR test to practice the production upgrade 
and backout, he had done the backout in the sandbox but wanted to see how 
long production would take.  He happens to be the tech with the least 
tenure as a tech but with many years of operations and batch support 
experience... sort of how a lot of people used to end up in tech, I guess. 
 

Thomas Ambros
Operating Systems and Connectivity Engineering
518-436-6433





From:   Jerry Whitteridge 
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   06/27/2012 12:48
Subject:Re: 'Inexperienced' RBS tech operative's blunder led to 
banking meltdown
Sent by:IBM Mainframe Discussion List 



One relates to a conditioning PTF required on the back level system to be 
able to backout an upgrade from a higher using a backup (DMPQ/MOVQ) 
process native to CA-7. As I read it they attempted to backout using the 
standard documented process but the downlevel system couldn't use the data 
formatted for the uplevel system and queue corruption might have occurred.

It's  shutting the door after the event - but also points to a code not a 
process problem.

Jerry Whitteridge
Lead Systems Programmer
Safeway Inc.
925 951 4184

If you feel in control
you just aren't going fast enough.


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On 
Behalf Of Bill Ashton
Sent: Wednesday, June 27, 2012 9:39 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking 
meltdown

Without having seen the HIPERs, it could be that they are trying to keep
others from blowing off their feet if they have staff who are
"operationally challenged."
B

On Wed, Jun 27, 2012 at 12:34 PM, Jerry Whitteridge <
jerry.whitteri...@safeway.com> wrote:

> I've just seen 3 HIPER PTF's from CA related to CA-7 that also fit the
> reported information on the RBS outage. I'd not be too hasty in blaming
> staff at this point.
>
> Jerry Whitteridge
> Lead Systems Programmer
> Safeway Inc.
> 925 951 4184
>
> If you feel in control
> you just aren't going fast enough.
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Knutson, Sam
> Sent: Wednesday, June 27, 2012 6:49 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking
> meltdown
>
> To someone who is inexperienced on platform I imagine managing a CA-7
> upgrade is like having the world's smartest dog but it only responds to
> commands in Latin.  Miscommunication is likely to be a source of
> dissatisfaction.
>
> The average person <30 is not stupid because they haven't been editing
> command decks and JCL for 25 years or lazy but software on our platform
> tends to be very particular and sometimes unforgiving of mistakes i.e.
> failure to specify QUEUE=NOFORMAT or some non-obvious phrase as part of
> START command.   Since business is no longer content to bring in newbies
> and let them spend 5-10 years to reach Journeyman status under the 
guidance
> of more senior folks software is going to have to get smarter and more
> forgiving, documentation is going to have to be written with less
> assumptions about the expertise of the "systems programmer" and zNextGen
> (generic) is going to operate "our" mainframes weather we think they are
> ready or not.  Some of those next generation folks are not going to be
> based in the home office anymore either whether we like that or not.
>
> The Register is fun reading sometimes but they rarely have enough detail
> to get real insight into what actually happened.  So really anything we 
say
> here is just idle speculation.   Feels like Friday so that's my .02 
worth
> of idle speculation :-)
>
> Best Regards,
>
> Sam Knutson, GEICO
> System z Team Leader
> mailto:sknut...@geico.com
> (office)  301.986.3574
> (cell) 301.996.1318
>
> "Think big, act bold, start simple, grow fast..."
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Tom Ambros
> Sent: Wednesday, June 27, 2012 8:25 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking
> meltdown
>
> "Complicated legacy mainframe system"?  It's CA-7!  If they think that's
> complicated they probably can't work a Mr. Coffee.
>
> Thomas Ambros
> Operating Systems and Connectivi

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Jerry Whitteridge]

One relates to a conditioning PTF required on the back level system to be able 
to backout an upgrade from a higher using a backup (DMPQ/MOVQ) process native 
to CA-7. As I read it they attempted to backout using the standard documented 
process but the downlevel system couldn't use the data formatted for the 
uplevel system and queue corruption might have occurred.

It's  shutting the door after the event - but also points to a code not a 
process problem.

Jerry Whitteridge
Lead Systems Programmer
Safeway Inc.
925 951 4184

If you feel in control
you just aren't going fast enough.


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Bill Ashton
Sent: Wednesday, June 27, 2012 9:39 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking 
meltdown

Without having seen the HIPERs, it could be that they are trying to keep
others from blowing off their feet if they have staff who are
"operationally challenged."
B

On Wed, Jun 27, 2012 at 12:34 PM, Jerry Whitteridge <
jerry.whitteri...@safeway.com> wrote:

> I've just seen 3 HIPER PTF's from CA related to CA-7 that also fit the
> reported information on the RBS outage. I'd not be too hasty in blaming
> staff at this point.
>
> Jerry Whitteridge
> Lead Systems Programmer
> Safeway Inc.
> 925 951 4184
>
> If you feel in control
> you just aren't going fast enough.
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Knutson, Sam
> Sent: Wednesday, June 27, 2012 6:49 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking
> meltdown
>
> To someone who is inexperienced on platform I imagine managing a CA-7
> upgrade is like having the world's smartest dog but it only responds to
> commands in Latin.  Miscommunication is likely to be a source of
> dissatisfaction.
>
> The average person <30 is not stupid because they haven't been editing
> command decks and JCL for 25 years or lazy but software on our platform
> tends to be very particular and sometimes unforgiving of mistakes i.e.
> failure to specify QUEUE=NOFORMAT or some non-obvious phrase as part of
> START command.   Since business is no longer content to bring in newbies
> and let them spend 5-10 years to reach Journeyman status under the guidance
> of more senior folks software is going to have to get smarter and more
> forgiving, documentation is going to have to be written with less
> assumptions about the expertise of the "systems programmer" and zNextGen
> (generic) is going to operate "our" mainframes weather we think they are
> ready or not.  Some of those next generation folks are not going to be
> based in the home office anymore either whether we like that or not.
>
> The Register is fun reading sometimes but they rarely have enough detail
> to get real insight into what actually happened.  So really anything we say
> here is just idle speculation.   Feels like Friday so that's my .02  worth
> of idle speculation :-)
>
> Best Regards,
>
> Sam Knutson, GEICO
> System z Team Leader
> mailto:sknut...@geico.com
> (office)  301.986.3574
> (cell) 301.996.1318
>
> "Think big, act bold, start simple, grow fast..."
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Tom Ambros
> Sent: Wednesday, June 27, 2012 8:25 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking
> meltdown
>
> "Complicated legacy mainframe system"?  It's CA-7!  If they think that's
> complicated they probably can't work a Mr. Coffee.
>
> Thomas Ambros
> Operating Systems and Connectivity Engineering
> 518-436-6433
>
>
>
>
>
> From:   Ed Gould 
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date:   06/27/2012 00:44
> Subject:'Inexperienced' RBS tech operative's blunder led to
> banking meltdown
> Sent by:IBM Mainframe Discussion List 
>
>
>
>
> http://www.theregister.co.uk/2012/06/26/rbs_natwest_ca_technologies_outsourcing/
>
>
>
> 
> This email/fax message is for the sole use of the intended
> recipient(s) and may contain confidential and privileged information.
> Any unauthorized review, use, disclosure or distribution of this
> email/fax is prohibited. If you are not the intended recipient, please
> destroy all paper

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Bill Ashton]

Without having seen the HIPERs, it could be that they are trying to keep
others from blowing off their feet if they have staff who are
"operationally challenged."
B

On Wed, Jun 27, 2012 at 12:34 PM, Jerry Whitteridge <
jerry.whitteri...@safeway.com> wrote:

> I've just seen 3 HIPER PTF's from CA related to CA-7 that also fit the
> reported information on the RBS outage. I'd not be too hasty in blaming
> staff at this point.
>
> Jerry Whitteridge
> Lead Systems Programmer
> Safeway Inc.
> 925 951 4184
>
> If you feel in control
> you just aren't going fast enough.
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Knutson, Sam
> Sent: Wednesday, June 27, 2012 6:49 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking
> meltdown
>
> To someone who is inexperienced on platform I imagine managing a CA-7
> upgrade is like having the world's smartest dog but it only responds to
> commands in Latin.  Miscommunication is likely to be a source of
> dissatisfaction.
>
> The average person <30 is not stupid because they haven't been editing
> command decks and JCL for 25 years or lazy but software on our platform
> tends to be very particular and sometimes unforgiving of mistakes i.e.
> failure to specify QUEUE=NOFORMAT or some non-obvious phrase as part of
> START command.   Since business is no longer content to bring in newbies
> and let them spend 5-10 years to reach Journeyman status under the guidance
> of more senior folks software is going to have to get smarter and more
> forgiving, documentation is going to have to be written with less
> assumptions about the expertise of the "systems programmer" and zNextGen
> (generic) is going to operate "our" mainframes weather we think they are
> ready or not.  Some of those next generation folks are not going to be
> based in the home office anymore either whether we like that or not.
>
> The Register is fun reading sometimes but they rarely have enough detail
> to get real insight into what actually happened.  So really anything we say
> here is just idle speculation.   Feels like Friday so that's my .02  worth
> of idle speculation :-)
>
> Best Regards,
>
> Sam Knutson, GEICO
> System z Team Leader
> mailto:sknut...@geico.com
> (office)  301.986.3574
> (cell) 301.996.1318
>
> "Think big, act bold, start simple, grow fast..."
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Tom Ambros
> Sent: Wednesday, June 27, 2012 8:25 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking
> meltdown
>
> "Complicated legacy mainframe system"?  It's CA-7!  If they think that's
> complicated they probably can't work a Mr. Coffee.
>
> Thomas Ambros
> Operating Systems and Connectivity Engineering
> 518-436-6433
>
>
>
>
>
> From:   Ed Gould 
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date:   06/27/2012 00:44
> Subject:'Inexperienced' RBS tech operative's blunder led to
> banking meltdown
> Sent by:IBM Mainframe Discussion List 
>
>
>
>
> http://www.theregister.co.uk/2012/06/26/rbs_natwest_ca_technologies_outsourcing/
>
>
>
> 
> This email/fax message is for the sole use of the intended
> recipient(s) and may contain confidential and privileged information.
> Any unauthorized review, use, disclosure or distribution of this
> email/fax is prohibited. If you are not the intended recipient, please
> destroy all paper and electronic copies of the original message.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>
>
> "Email Firewall" made the following annotations.
>
> --
>
> Warning:
> All e-mail sent to this address will be received by the corporate e-mail
> system, and is subject to archival and review by someone other than the
> recipient.  This e-mail may contain proprietary information and is intended
> only for the use of the intended recipient(s).  If the reader of this
> message is not the intended recipient(s), you are notified that you have
> received this message in error and that any rev

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Jerry Whitteridge]

I've just seen 3 HIPER PTF's from CA related to CA-7 that also fit the reported 
information on the RBS outage. I'd not be too hasty in blaming staff at this 
point.

Jerry Whitteridge
Lead Systems Programmer
Safeway Inc.
925 951 4184

If you feel in control
you just aren't going fast enough.


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Knutson, Sam
Sent: Wednesday, June 27, 2012 6:49 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking 
meltdown

To someone who is inexperienced on platform I imagine managing a CA-7 upgrade 
is like having the world's smartest dog but it only responds to commands in 
Latin.  Miscommunication is likely to be a source of dissatisfaction.

The average person <30 is not stupid because they haven't been editing command 
decks and JCL for 25 years or lazy but software on our platform tends to be 
very particular and sometimes unforgiving of mistakes i.e. failure to specify 
QUEUE=NOFORMAT or some non-obvious phrase as part of START command.   Since 
business is no longer content to bring in newbies and let them spend 5-10 years 
to reach Journeyman status under the guidance of more senior folks software is 
going to have to get smarter and more forgiving, documentation is going to have 
to be written with less assumptions about the expertise of the "systems 
programmer" and zNextGen (generic) is going to operate "our" mainframes weather 
we think they are ready or not.  Some of those next generation folks are not 
going to be based in the home office anymore either whether we like that or 
not.  

The Register is fun reading sometimes but they rarely have enough detail to get 
real insight into what actually happened.  So really anything we say here is 
just idle speculation.   Feels like Friday so that's my .02  worth of idle 
speculation :-)   

    Best Regards, 

    Sam Knutson, GEICO 
    System z Team Leader 
    mailto:sknut...@geico.com 
    (office)  301.986.3574 
    (cell) 301.996.1318   
   
"Think big, act bold, start simple, grow fast..." 


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Tom Ambros
Sent: Wednesday, June 27, 2012 8:25 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking 
meltdown

"Complicated legacy mainframe system"?  It's CA-7!  If they think that's 
complicated they probably can't work a Mr. Coffee. 

Thomas Ambros
Operating Systems and Connectivity Engineering
518-436-6433





From:   Ed Gould 
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   06/27/2012 00:44
Subject:'Inexperienced' RBS tech operative's blunder led to 
banking meltdown
Sent by:IBM Mainframe Discussion List 



http://www.theregister.co.uk/2012/06/26/rbs_natwest_ca_technologies_outsourcing/
 




This email/fax message is for the sole use of the intended
recipient(s) and may contain confidential and privileged information.
Any unauthorized review, use, disclosure or distribution of this
email/fax is prohibited. If you are not the intended recipient, please
destroy all paper and electronic copies of the original message.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


"Email Firewall" made the following annotations.
--

Warning: 
All e-mail sent to this address will be received by the corporate e-mail 
system, and is subject to archival and review by someone other than the 
recipient.  This e-mail may contain proprietary information and is intended 
only for the use of the intended recipient(s).  If the reader of this message 
is not the intended recipient(s), you are notified that you have received this 
message in error and that any review, dissemination, distribution or copying of 
this message is strictly prohibited.  If you have received this message in 
error, please notify the sender immediately.   
 
==

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Dave Salt]

> From: john.mck...@healthmarkets.com
>I wonder if anybody has 
done a study of productivity between "old style" development using ISPF 
and edit-compile-test versus using the RD/z Eclipse based software.
> 

Well if they did I hope they at least used SimpList or IPT instead of vanilla 
ISPF, otherwise they'd be painting the mainframe in a very unproductive and 
unfair light.
  
Dave Salt

SimpList(tm) - try it; you'll get it! 

http://www.mackinney.com/products/program-development/simplist.html  









  
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Dave Day]

Bill,

Its more than just a specific transaction by an individual being 
committed.  A lot of it has to do with clearing house work.  If you 
write a check to someone, that someone will eventually deposit it in 
their bank.  That might not be the same bank that you use.  It then has 
to go thru a clearing house.  The clearing house(s) used to work on a 
time schedule, wherein all transactions for a preceding time period 
would be batched up and processed, and distributed at a specific time on 
a normal business day.  Same thing holds true for payroll updates, gov't 
checks, and a myriad of other types of transactions.


--Dave
On 6/27/2012 10:30 AM, Bill Fairchild wrote:

I understand why all accumulated transactions have to be processed in the 
correct sequence if they are batched up daily, but why is an individual 
transaction not instantaneously and permanently processed with the results 
being committed right then?  Why is it necessary to run huge batches of 
transactions at night?   I am guessing that committing the transaction 
instantly would take too long for the average customer's patience when standing 
at an outdoors ATM in the rain.  Are there any other reasons?  When I buy an 
airplane ticket online, I get my whole transaction complete in one sitting and 
do not have to wait until the next day to receive confirmation of my 
reservation, but it always takes several minutes to do it online.

Bill Fairchild
Programmer
Rocket Software
408 Chamberlain Park Lane * Franklin, TN 37069-2526 * USA
t: +1.617.614.4503 *  e: bfairch...@rocketsoftware.com * w: 
www.rocketsoftware.com


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of J. Cassidy
Sent: Wednesday, June 27, 2012 9:58 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking 
meltdown

The Guardian has a version that is more or less readable, albeit meant for the 
man on the street.

Some of the comments are  rather "pungent".

http://www.guardian.co.uk/technology/2012/jun/25/how-natwest-it-meltdown?newsfeed=true

=> On Wed, 27 Jun 2012 09:49:03 -0400, Knutson, Sam  => 
wrote:
=>
=> 
=>
=> Well put Sam.
=>
=>>
=>>The Register is fun reading sometimes but they rarely have enough detail =>> to 
get real insight into what actually happened.  So really anything we =>> say here is just 
idle speculation.
=>>
=>
=> Exactly what I told an old manager of mine who emailed me the
=> story after it started circulating at his office.   I told him to think
=> of it as the "National Enquirer of IT".  :-) => => Mark => -- => Mark Zelden - 
Zelden Consulting Services - z/OS, OS/390 and MVS => mailto:m...@mzelden.com => Mark's MVS Utilities: 
http://www.mzelden.com/mvsutil.html
=> Systems Programming expert at http://expertanswercenter.techtarget.com/
=>
=> --
=> For IBM-MAIN subscribe / signoff / archive access instructions, => send email 
to lists...@listserv.ua.edu with the message: INFO IBM-MAIN =>


John Cassidy (Dipl.-Ingr.)

Kapellenstr. 21a

D-65193 Wiesbaden

EU



Mobile: +49 (0) 170 794 3616


http://www.JDCassidy.net

http://en.federaleurope.org/

http://sva-zhosting.com/en/index.php

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Phil Smith]

Bill Fairchild wrote:
>I understand why all accumulated transactions have to be processed in the 
>correct sequence if they are batched up daily, but why is an individual 
>transaction not instantaneously and permanently processed with the results 
>being committed right then?  Why is it necessary to run huge batches of 
>transactions at night?   I am guessing that committing the transaction 
>instantly would take too long for the average customer's patience when 
>standing at an outdoors ATM in the rain.  Are there any other reasons?  When I 
>buy an airplane ticket online, I get my whole transaction complete in one 
>sitting and do not have to wait until the next day to receive confirmation of 
>my reservation, but it always takes several minutes to do it online.

Besides/in addition to Mike Schwab's point, credit and signature debit 
transactions aren't processed in real-time: an authorization is processed, but 
the actual charge/debit doesn't go through until settlement.

Come to session 11409: The Payments Ecosystem: Security Challenges in the 
21st Century 
at SHARE in Anaheim, Monday at 3PM, to learn more about how this
--
...phsiii

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Rob Schramm]

http://cs.ua.edu/~SECSE08/Papers/Danis.pdf

Rob Schramm
Senior Systems Consultant
Imperium Group




On Wed, Jun 27, 2012 at 11:43 AM, Phil Smith  wrote:

> John McKown wrote:
> >I somewhat agree (as difficult as that is for me). In today's world, it
> is expected that a worker will just walk in off the street and have the
> "intuitive" knowledge of how to use computers. As much as I dislike it
> personally, z/OS really needs the "new look" interface to present to the
> end users, and even programmers. I wonder if anybody has done a study of
> productivity between "old style" development using ISPF and
> edit-compile-test versus using the RD/z Eclipse based software.
>
> Of course they have...many times. And they all show that IDEs are more
> productive, FVSO "more productive": programs get done faster. Not
> necessarily better, but faster. And it's a positive feedback loop: once the
> IDE is all anyone has used, old-school programming becomes unthinkable.
>
> And that's how we get the instability that is Windows. Not even
> necessarily the Microsoft end of it: a near-infinite number of vendors with
> an infinite number of monkeys banging away at their IDEs, producing
> products that sort of work, but perhaps destabilize the underlying OS (or
> Office, or some other product) may be the real cause.
>
> Of course this is an oversimplification, but when you make programming so
> simple, even a kid can do it, you get programs written by kids.
>
> Even open source, for all its benefits, isn't a real solution: look at the
> plugin-container hack Firefox has resorted to. Is that strictly because
> Flash is so unstable? Chrome doesn't seem to have any significant Flash
> problems (or maybe I just use Firefox more). I'm on the Beta channel of
> Firefox right now because the production version was hanging my Windows 7
> machines (maybe that's been fixed by now, but it *was* true). Worse, I had
> no real way to debug it.
>
> OK, so I sound like a cranky old fart. If the shoe fits...
> --
> ...phsiii
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Darth Keller]

>>  And whoever decided Tuesday night was the most appropriate probably 
has 
>>  some explaining to do.  Right before a full sized cycle?  Unless every 

>>  night is like that, probably not the most prudent decision. 

>> Thomas Ambros

Agreed - I'd be curious to know how this got through any kind of 
formalized Change Control Process. 
ddk

This e-mail message and all attachments transmitted with it may
contain legally privileged and/or confidential information intended
solely for the use of the addressee(s). If the reader of this
message is not the intended recipient, you are hereby notified that
any reading, dissemination, distribution, copying, forwarding or
other use of this message or its attachments is strictly
prohibited. If you have received this message in error, please
notify the sender immediately and delete this message and all
copies and backups thereof. Thank you.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Mike Schwab]

Banking rules say you have to process the day's deposits before the
day's withdrawals.

Basically, you post the transaction to a transaction file which is
applied to the account after the cutoff time.

On Wed, Jun 27, 2012 at 10:30 AM, Bill Fairchild
 wrote:
> I understand why all accumulated transactions have to be processed in the 
> correct sequence if they are batched up daily, but why is an individual 
> transaction not instantaneously and permanently processed with the results 
> being committed right then?  Why is it necessary to run huge batches of 
> transactions at night?   I am guessing that committing the transaction 
> instantly would take too long for the average customer's patience when 
> standing at an outdoors ATM in the rain.  Are there any other reasons?  When 
> I buy an airplane ticket online, I get my whole transaction complete in one 
> sitting and do not have to wait until the next day to receive confirmation of 
> my reservation, but it always takes several minutes to do it online.
>
-- 
Mike A Schwab, Springfield IL USA
Where do Forest Rangers go to get away from it all?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Phil Smith]

John McKown wrote:
>I somewhat agree (as difficult as that is for me). In today's world, it is 
>expected that a worker will just walk in off the street and have the 
>"intuitive" knowledge of how to use computers. As much as I dislike it 
>personally, z/OS really needs the "new look" interface to present to the end 
>users, and even programmers. I wonder if anybody has done a study of 
>productivity between "old style" development using ISPF and edit-compile-test 
>versus using the RD/z Eclipse based software.

Of course they have...many times. And they all show that IDEs are more 
productive, FVSO "more productive": programs get done faster. Not necessarily 
better, but faster. And it's a positive feedback loop: once the IDE is all 
anyone has used, old-school programming becomes unthinkable.

And that's how we get the instability that is Windows. Not even necessarily the 
Microsoft end of it: a near-infinite number of vendors with an infinite number 
of monkeys banging away at their IDEs, producing products that sort of work, 
but perhaps destabilize the underlying OS (or Office, or some other product) 
may be the real cause.

Of course this is an oversimplification, but when you make programming so 
simple, even a kid can do it, you get programs written by kids.

Even open source, for all its benefits, isn't a real solution: look at the 
plugin-container hack Firefox has resorted to. Is that strictly because Flash 
is so unstable? Chrome doesn't seem to have any significant Flash problems (or 
maybe I just use Firefox more). I'm on the Beta channel of Firefox right now 
because the production version was hanging my Windows 7 machines (maybe that's 
been fixed by now, but it *was* true). Worse, I had no real way to debug it.

OK, so I sound like a cranky old fart. If the shoe fits...
--
...phsiii

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Tom Ambros]

And whoever decided Tuesday night was the most appropriate probably has 
some explaining to do.  Right before a full sized cycle?  Unless every 
night is like that, probably not the most prudent decision. 

Thomas Ambros
Operating Systems and Connectivity Engineering
518-436-6433





From:   "McKown, John" 
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   06/27/2012 11:26
Subject:    Re: 'Inexperienced' RBS tech operative's blunder led to 
banking meltdown
Sent by:IBM Mainframe Discussion List 



I somewhat agree (as difficult as that is for me). In today's world, it is 
expected that a worker will just walk in off the street and have the 
"intuitive" knowledge of how to use computers. As much as I dislike it 
personally, z/OS really needs the "new look" interface to present to the 
end users, and even programmers. I wonder if anybody has done a study of 
productivity between "old style" development using ISPF and 
edit-compile-test versus using the RD/z Eclipse based software.

--
John McKown 
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets®

9151 Boulevard 26 . N. Richland Hills . TX 76010
(817) 255-3225 phone . 
john.mck...@healthmarkets.com . www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or 
proprietary information. If you are not the intended recipient, please 
contact the sender by reply e-mail and destroy all copies of the original 
message. HealthMarkets® is the brand name for products underwritten and 
issued by the insurance subsidiaries of HealthMarkets, Inc. -The 
Chesapeake Life Insurance Company®, Mid-West National Life Insurance 
Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM

 

> -Original Message-
> From: IBM Mainframe Discussion List 
> [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Knutson, Sam
> Sent: Wednesday, June 27, 2012 8:49 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led 
> to banking meltdown
> 
> To someone who is inexperienced on platform I imagine 
> managing a CA-7 upgrade is like having the world's smartest 
> dog but it only responds to commands in Latin. 
> Miscommunication is likely to be a source of dissatisfaction.
> 
> The average person <30 is not stupid because they haven't 
> been editing command decks and JCL for 25 years or lazy but 
> software on our platform tends to be very particular and 
> sometimes unforgiving of mistakes i.e. failure to specify 
> QUEUE=NOFORMAT or some non-obvious phrase as part of START 
> command.   Since business is no longer content to bring in 
> newbies and let them spend 5-10 years to reach Journeyman 
> status under the guidance of more senior folks software is 
> going to have to get smarter and more forgiving, 
> documentation is going to have to be written with less 
> assumptions about the expertise of the "systems programmer" 
> and zNextGen (generic) is going to operate "our" mainframes 
> weather we think they are ready or not.  Some of those next 
> generation folks are not going to be based in the home office 
> anymore either whether we like that or not. 
> 
> The Register is fun reading sometimes but they rarely have 
> enough detail to get real insight into what actually 
> happened.  So really anything we say here is just idle 
> speculation.   Feels like Friday so that's my .02  worth of 
> idle speculation :-) 
> 
> Best Regards, 
> 
> Sam Knutson, GEICO 
> System z Team Leader 
> mailto:sknut...@geico.com 
> (office)  301.986.3574 
> (cell) 301.996.1318   
>
> "Think big, act bold, start simple, grow fast..." 
> 
> 
> -----Original Message-
> From: IBM Mainframe Discussion List 
> [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Tom Ambros
> Sent: Wednesday, June 27, 2012 8:25 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led 
> to banking meltdown
> 
> "Complicated legacy mainframe system"?  It's CA-7!  If they 
> think that's complicated they probably can't work a Mr. Coffee. 
> 
> Thomas Ambros
> Operating Systems and Connectivity Engineering
> 518-436-6433
> 
> 
> 
> 
> 
> From:   Ed Gould 
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date:   06/27/2012 00:44
> Subject:'Inexperienced' RBS tech operative's blunder led to 
> banking meltdown
> Sent by:IBM Mainframe Discussion List 
> 
> 
> 
> 
> http://www.theregister.co.uk/2012/06/26/rbs_natwe

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Bill Fairchild]

I understand why all accumulated transactions have to be processed in the 
correct sequence if they are batched up daily, but why is an individual 
transaction not instantaneously and permanently processed with the results 
being committed right then?  Why is it necessary to run huge batches of 
transactions at night?   I am guessing that committing the transaction 
instantly would take too long for the average customer's patience when standing 
at an outdoors ATM in the rain.  Are there any other reasons?  When I buy an 
airplane ticket online, I get my whole transaction complete in one sitting and 
do not have to wait until the next day to receive confirmation of my 
reservation, but it always takes several minutes to do it online.

Bill Fairchild
Programmer
Rocket Software
408 Chamberlain Park Lane * Franklin, TN 37069-2526 * USA
t: +1.617.614.4503 *  e: bfairch...@rocketsoftware.com * w: 
www.rocketsoftware.com


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of J. Cassidy
Sent: Wednesday, June 27, 2012 9:58 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking 
meltdown

The Guardian has a version that is more or less readable, albeit meant for the 
man on the street.

Some of the comments are  rather "pungent".

http://www.guardian.co.uk/technology/2012/jun/25/how-natwest-it-meltdown?newsfeed=true

=> On Wed, 27 Jun 2012 09:49:03 -0400, Knutson, Sam  => 
wrote:
=>
=> 
=>
=> Well put Sam.
=>
=>>
=>>The Register is fun reading sometimes but they rarely have enough detail =>> 
to get real insight into what actually happened.  So really anything we =>> say 
here is just idle speculation.
=>>
=>
=> Exactly what I told an old manager of mine who emailed me the
=> story after it started circulating at his office.   I told him to think
=> of it as the "National Enquirer of IT".  :-) => => Mark => -- => Mark Zelden 
- Zelden Consulting Services - z/OS, OS/390 and MVS => mailto:m...@mzelden.com 
=> Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html
=> Systems Programming expert at http://expertanswercenter.techtarget.com/
=>
=> --
=> For IBM-MAIN subscribe / signoff / archive access instructions, => send 
email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN =>


John Cassidy (Dipl.-Ingr.)

Kapellenstr. 21a

D-65193 Wiesbaden

EU



Mobile: +49 (0) 170 794 3616


http://www.JDCassidy.net

http://en.federaleurope.org/

http://sva-zhosting.com/en/index.php

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[McKown, John]

I somewhat agree (as difficult as that is for me). In today's world, it is 
expected that a worker will just walk in off the street and have the 
"intuitive" knowledge of how to use computers. As much as I dislike it 
personally, z/OS really needs the "new look" interface to present to the end 
users, and even programmers. I wonder if anybody has done a study of 
productivity between "old style" development using ISPF and edit-compile-test 
versus using the RD/z Eclipse based software.

--
John McKown 
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets®

9151 Boulevard 26 . N. Richland Hills . TX 76010
(817) 255-3225 phone . 
john.mck...@healthmarkets.com . www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or 
proprietary information. If you are not the intended recipient, please contact 
the sender by reply e-mail and destroy all copies of the original message. 
HealthMarkets® is the brand name for products underwritten and issued by the 
insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance 
Company®, Mid-West National Life Insurance Company of TennesseeSM and The MEGA 
Life and Health Insurance Company.SM

 

> -Original Message-
> From: IBM Mainframe Discussion List 
> [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Knutson, Sam
> Sent: Wednesday, June 27, 2012 8:49 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led 
> to banking meltdown
> 
> To someone who is inexperienced on platform I imagine 
> managing a CA-7 upgrade is like having the world's smartest 
> dog but it only responds to commands in Latin.  
> Miscommunication is likely to be a source of dissatisfaction.
> 
> The average person <30 is not stupid because they haven't 
> been editing command decks and JCL for 25 years or lazy but 
> software on our platform tends to be very particular and 
> sometimes unforgiving of mistakes i.e. failure to specify 
> QUEUE=NOFORMAT or some non-obvious phrase as part of START 
> command.   Since business is no longer content to bring in 
> newbies and let them spend 5-10 years to reach Journeyman 
> status under the guidance of more senior folks software is 
> going to have to get smarter and more forgiving, 
> documentation is going to have to be written with less 
> assumptions about the expertise of the "systems programmer" 
> and zNextGen (generic) is going to operate "our" mainframes 
> weather we think they are ready or not.  Some of those next 
> generation folks are not going to be based in the home office 
> anymore either whether we like that or not.  
> 
> The Register is fun reading sometimes but they rarely have 
> enough detail to get real insight into what actually 
> happened.  So really anything we say here is just idle 
> speculation.   Feels like Friday so that's my .02  worth of 
> idle speculation :-)   
> 
>     Best Regards, 
> 
>     Sam Knutson, GEICO 
>     System z Team Leader 
>     mailto:sknut...@geico.com 
>     (office)  301.986.3574 
>     (cell) 301.996.1318   
>    
> "Think big, act bold, start simple, grow fast..." 
> 
> 
> -Original Message-
> From: IBM Mainframe Discussion List 
> [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Tom Ambros
> Sent: Wednesday, June 27, 2012 8:25 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led 
> to banking meltdown
> 
> "Complicated legacy mainframe system"?  It's CA-7!  If they 
> think that's complicated they probably can't work a Mr. Coffee. 
> 
> Thomas Ambros
> Operating Systems and Connectivity Engineering
> 518-436-6433
> 
> 
> 
> 
> 
> From:   Ed Gould 
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date:   06/27/2012 00:44
> Subject:'Inexperienced' RBS tech operative's blunder led to 
> banking meltdown
> Sent by:IBM Mainframe Discussion List 
> 
> 
> 
> 
> http://www.theregister.co.uk/2012/06/26/rbs_natwest_ca_technol
> ogies_outsourcing/ 
> 
> 
> 
> 
> This email/fax message is for the sole use of the intended
> recipient(s) and may contain confidential and privileged information.
> Any unauthorized review, use, disclosure or distribution of this
> email/fax is prohibited. If you are not the intended recipient, please
> destroy all paper and electronic copies of the original message.
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> 
> 
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Anne & Lynn Wheeler]

thomas_amb...@keybank.com (Tom Ambros) writes:
> "Complicated legacy mainframe system"?  It's CA-7!  If they think that's 
> complicated they probably can't work a Mr. Coffee. 

Key questions on the massive RBS / NatWest IT failure
http://www.zdnet.com/blog/projectfailures/key-questions-on-the-massive-rbs-natwest-it-failure/15706

references

RBS Bank joins the IT failures 'Hall of Shame'
http://www.zdnet.com/blog/projectfailures/rbs-bank-joins-the-it-failures-hall-of-shame/15685
Guardian's investigations suggest bank's problems began on Tuesday night
when it updated key piece of software called CA-7
http://www.guardian.co.uk/technology/2012/jun/25/how-natwest-it-meltdown

and

Update 6/27/12: According the The Register (a sensationalist tech news
site), an inexperienced computer operator in India caused the RBS
failure. Apparently, a routine problem arose during the upgrade
procedure, which is usually not a serious issue because administrators
can roll back to a previous and stable version of the software. In this
case, however, it seems the operator erroneously cleared the entire
transaction job queue, kicking off a long and difficult process of
reconstruction. The article adds: A complicated legacy
mainframe system at RBS and a team inexperienced in its quirks made the
problem harder to fix.

... snip ...

Inexperienced' RBS tech operative's blunder led to banking meltdown
http://www.theregister.co.uk/2012/06/26/rbs_natwest_ca_technologies_outsourcing/



-- 
virtualization experience starting Jan1968, online at home since Mar1970

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[J. Cassidy]

The Guardian has a version that is more or less readable, albeit meant for
the man on the street.

Some of the comments are  rather "pungent".

http://www.guardian.co.uk/technology/2012/jun/25/how-natwest-it-meltdown?newsfeed=true

=> On Wed, 27 Jun 2012 09:49:03 -0400, Knutson, Sam 
=> wrote:
=>
=> 
=>
=> Well put Sam.
=>
=>>
=>>The Register is fun reading sometimes but they rarely have enough detail
=>> to get real insight into what actually happened.  So really anything we
=>> say here is just idle speculation.
=>>
=>
=> Exactly what I told an old manager of mine who emailed me the
=> story after it started circulating at his office.   I told him to think
=> of it as the "National Enquirer of IT".  :-)
=>
=> Mark
=> --
=> Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS
=> mailto:m...@mzelden.com
=> Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html
=> Systems Programming expert at http://expertanswercenter.techtarget.com/
=>
=> --
=> For IBM-MAIN subscribe / signoff / archive access instructions,
=> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
=>


John Cassidy (Dipl.-Ingr.)

Kapellenstr. 21a

D-65193 Wiesbaden

EU



Mobile: +49 (0) 170 794 3616


http://www.JDCassidy.net

http://en.federaleurope.org/

http://sva-zhosting.com/en/index.php

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Mark Zelden]

On Wed, 27 Jun 2012 09:49:03 -0400, Knutson, Sam  wrote:



Well put Sam.

>
>The Register is fun reading sometimes but they rarely have enough detail to 
>get real insight into what actually happened.  So really anything we say here 
>is just idle speculation.
>

Exactly what I told an old manager of mine who emailed me the
story after it started circulating at his office.   I told him to think
of it as the "National Enquirer of IT".  :-)

Mark
--
Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS   
mailto:m...@mzelden.com
Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html 
Systems Programming expert at http://expertanswercenter.techtarget.com/

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Knutson, Sam]

To someone who is inexperienced on platform I imagine managing a CA-7 upgrade 
is like having the world's smartest dog but it only responds to commands in 
Latin.  Miscommunication is likely to be a source of dissatisfaction.

The average person <30 is not stupid because they haven't been editing command 
decks and JCL for 25 years or lazy but software on our platform tends to be 
very particular and sometimes unforgiving of mistakes i.e. failure to specify 
QUEUE=NOFORMAT or some non-obvious phrase as part of START command.   Since 
business is no longer content to bring in newbies and let them spend 5-10 years 
to reach Journeyman status under the guidance of more senior folks software is 
going to have to get smarter and more forgiving, documentation is going to have 
to be written with less assumptions about the expertise of the "systems 
programmer" and zNextGen (generic) is going to operate "our" mainframes weather 
we think they are ready or not.  Some of those next generation folks are not 
going to be based in the home office anymore either whether we like that or 
not.  

The Register is fun reading sometimes but they rarely have enough detail to get 
real insight into what actually happened.  So really anything we say here is 
just idle speculation.   Feels like Friday so that's my .02  worth of idle 
speculation :-)   

    Best Regards, 

    Sam Knutson, GEICO 
    System z Team Leader 
    mailto:sknut...@geico.com 
    (office)  301.986.3574 
    (cell) 301.996.1318   
   
"Think big, act bold, start simple, grow fast..." 


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Tom Ambros
Sent: Wednesday, June 27, 2012 8:25 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking 
meltdown

"Complicated legacy mainframe system"?  It's CA-7!  If they think that's 
complicated they probably can't work a Mr. Coffee. 

Thomas Ambros
Operating Systems and Connectivity Engineering
518-436-6433





From:   Ed Gould 
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   06/27/2012 00:44
Subject:'Inexperienced' RBS tech operative's blunder led to 
banking meltdown
Sent by:IBM Mainframe Discussion List 



http://www.theregister.co.uk/2012/06/26/rbs_natwest_ca_technologies_outsourcing/
 




This email/fax message is for the sole use of the intended
recipient(s) and may contain confidential and privileged information.
Any unauthorized review, use, disclosure or distribution of this
email/fax is prohibited. If you are not the intended recipient, please
destroy all paper and electronic copies of the original message.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[John Gilmore]

On 6/27/12, Scott Ford  wrote:
> John,
>
> Count me in...
>
> Scott ford
> www.identityforge.com
>
> On Jun 27, 2012, at 8:33 AM, "McKown, John" 
> wrote:
>
>> If it's not "point and click", then it is beyond today's average person. I
>> have envisioned a way to make so much money, I could bail out the
>> Eurozone. Make a UI which is modelled on the FRPG "first person shooter"
>> interfaces. "Frag that accounts receivable information!". After all, it is
>> what the majority of kids today are learning. "Accounting for Nintendo DS
>> User" would be a best seller (audio format - rap style, but with optional
>> epub or mobi, for the few who can read)
>>
>> --
>> John McKown
>> Systems Engineer IV
>> IT
>>
>> Administrative Services Group
>>
>> HealthMarkets(r)
>>
>> 9151 Boulevard 26 * N. Richland Hills * TX 76010
>> (817) 255-3225 phone *
>> john.mck...@healthmarkets.com * www.HealthMarkets.com
>>
>> Confidentiality Notice: This e-mail message may contain confidential or
>> proprietary information. If you are not the intended recipient, please
>> contact the sender by reply e-mail and destroy all copies of the original
>> message. HealthMarkets(r) is the brand name for products underwritten and
>> issued by the insurance subsidiaries of HealthMarkets, Inc. -The
>> Chesapeake Life Insurance Company(r), Mid-West National Life Insurance
>> Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM
>>
>>> -----Original Message-
>>> From: IBM Mainframe Discussion List
>>> [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Tom Ambros
>>> Sent: Wednesday, June 27, 2012 7:25 AM
>>> To: IBM-MAIN@LISTSERV.UA.EDU
>>> Subject: Re: 'Inexperienced' RBS tech operative's blunder led
>>> to banking meltdown
>>>
>>> "Complicated legacy mainframe system"?  It's CA-7!  If they
>>> think that's
>>> complicated they probably can't work a Mr. Coffee.
>>>
>>> Thomas Ambros
>>> Operating Systems and Connectivity Engineering
>>> 518-436-6433
>>>
>>>
>>>
>>>
>>>
>>> From:   Ed Gould 
>>> To: IBM-MAIN@LISTSERV.UA.EDU
>>> Date:   06/27/2012 00:44
>>> Subject:'Inexperienced' RBS tech operative's blunder led to
>>> banking meltdown
>>> Sent by:IBM Mainframe Discussion List
>>> 
>>>
>>>
>>>
>>> http://www.theregister.co.uk/2012/06/26/
>>> rbs_natwest_ca_technologies_outsourcing/
>>>
>>>
>>> --
>>> For IBM-MAIN subscribe / signoff / archive access instructions,
>>> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>>>
>>> This communication may contain privileged and/or confidential
>>> information. It is intended solely for the use of the
>>> addressee. If you are not the intended recipient, you are
>>> strictly prohibited from disclosing, copying, distributing or
>>> using any of this information. If you received this
>>> communication in error, please contact the sender immediately
>>> and destroy the material in its entirety, whether electronic
>>> or hard copy. This communication may contain nonpublic
>>> personal information about consumers subject to the
>>> restrictions of the Gramm-Leach-Bliley Act. You may not
>>> directly or indirectly reuse or redisclose such information
>>> for any purpose other than to provide the services for which
>>> you are receiving the information.
>>>
>>> 127 Public Square, Cleveland, OH 44114
>>> If you prefer not to receive future e-mail offers for
>>> products or services from Key
>>> send an e-mail to mailto:dnereque...@key.com with 'No
>>> Promotional E-mails' in the
>>> SUBJECT line.
>>>
>>> --
>>> For IBM-MAIN subscribe / signoff / archive access instructions,
>>> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>>>
>>>
>> --
>> For IBM-MAIN subscribe / signoff / archive access instructions,
>> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Scott Ford]

John,

Count me in...

Scott ford
www.identityforge.com

On Jun 27, 2012, at 8:33 AM, "McKown, John"  
wrote:

> If it's not "point and click", then it is beyond today's average person. I 
> have envisioned a way to make so much money, I could bail out the Eurozone. 
> Make a UI which is modelled on the FRPG "first person shooter" interfaces. 
> "Frag that accounts receivable information!". After all, it is what the 
> majority of kids today are learning. "Accounting for Nintendo DS User" would 
> be a best seller (audio format - rap style, but with optional epub or mobi, 
> for the few who can read)
> 
> -- 
> John McKown 
> Systems Engineer IV
> IT
> 
> Administrative Services Group
> 
> HealthMarkets(r)
> 
> 9151 Boulevard 26 * N. Richland Hills * TX 76010
> (817) 255-3225 phone * 
> john.mck...@healthmarkets.com * www.HealthMarkets.com
> 
> Confidentiality Notice: This e-mail message may contain confidential or 
> proprietary information. If you are not the intended recipient, please 
> contact the sender by reply e-mail and destroy all copies of the original 
> message. HealthMarkets(r) is the brand name for products underwritten and 
> issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake 
> Life Insurance Company(r), Mid-West National Life Insurance Company of 
> TennesseeSM and The MEGA Life and Health Insurance Company.SM
> 
>> -Original Message-
>> From: IBM Mainframe Discussion List 
>> [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Tom Ambros
>> Sent: Wednesday, June 27, 2012 7:25 AM
>> To: IBM-MAIN@LISTSERV.UA.EDU
>> Subject: Re: 'Inexperienced' RBS tech operative's blunder led 
>> to banking meltdown
>> 
>> "Complicated legacy mainframe system"?  It's CA-7!  If they 
>> think that's 
>> complicated they probably can't work a Mr. Coffee. 
>> 
>> Thomas Ambros
>> Operating Systems and Connectivity Engineering
>> 518-436-6433
>> 
>> 
>> 
>> 
>> 
>> From:   Ed Gould 
>> To: IBM-MAIN@LISTSERV.UA.EDU
>> Date:   06/27/2012 00:44
>> Subject:'Inexperienced' RBS tech operative's blunder led to 
>> banking meltdown
>> Sent by:IBM Mainframe Discussion List 
>> 
>> 
>> 
>> 
>> http://www.theregister.co.uk/2012/06/26/ 
>> rbs_natwest_ca_technologies_outsourcing/
>> 
>> 
>> --
>> For IBM-MAIN subscribe / signoff / archive access instructions,
>> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>> 
>> This communication may contain privileged and/or confidential 
>> information. It is intended solely for the use of the 
>> addressee. If you are not the intended recipient, you are 
>> strictly prohibited from disclosing, copying, distributing or 
>> using any of this information. If you received this 
>> communication in error, please contact the sender immediately 
>> and destroy the material in its entirety, whether electronic 
>> or hard copy. This communication may contain nonpublic 
>> personal information about consumers subject to the 
>> restrictions of the Gramm-Leach-Bliley Act. You may not 
>> directly or indirectly reuse or redisclose such information 
>> for any purpose other than to provide the services for which 
>> you are receiving the information.
>> 
>> 127 Public Square, Cleveland, OH 44114
>> If you prefer not to receive future e-mail offers for 
>> products or services from Key 
>> send an e-mail to mailto:dnereque...@key.com with 'No 
>> Promotional E-mails' in the 
>> SUBJECT line.
>> 
>> --
>> For IBM-MAIN subscribe / signoff / archive access instructions,
>> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>> 
>> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[McKown, John]

If it's not "point and click", then it is beyond today's average person. I have 
envisioned a way to make so much money, I could bail out the Eurozone. Make a 
UI which is modelled on the FRPG "first person shooter" interfaces. "Frag that 
accounts receivable information!". After all, it is what the majority of kids 
today are learning. "Accounting for Nintendo DS User" would be a best seller 
(audio format - rap style, but with optional epub or mobi, for the few who can 
read)

-- 
John McKown 
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets(r)

9151 Boulevard 26 * N. Richland Hills * TX 76010
(817) 255-3225 phone * 
john.mck...@healthmarkets.com * www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or 
proprietary information. If you are not the intended recipient, please contact 
the sender by reply e-mail and destroy all copies of the original message. 
HealthMarkets(r) is the brand name for products underwritten and issued by the 
insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance 
Company(r), Mid-West National Life Insurance Company of TennesseeSM and The 
MEGA Life and Health Insurance Company.SM

> -Original Message-
> From: IBM Mainframe Discussion List 
> [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Tom Ambros
> Sent: Wednesday, June 27, 2012 7:25 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led 
> to banking meltdown
> 
> "Complicated legacy mainframe system"?  It's CA-7!  If they 
> think that's 
> complicated they probably can't work a Mr. Coffee. 
> 
> Thomas Ambros
> Operating Systems and Connectivity Engineering
> 518-436-6433
> 
> 
> 
> 
> 
> From:   Ed Gould 
> To: IBM-MAIN@LISTSERV.UA.EDU
> Date:   06/27/2012 00:44
> Subject:'Inexperienced' RBS tech operative's blunder led to 
> banking meltdown
> Sent by:IBM Mainframe Discussion List 
> 
> 
> 
> 
> http://www.theregister.co.uk/2012/06/26/ 
> rbs_natwest_ca_technologies_outsourcing/
> 
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> 
> This communication may contain privileged and/or confidential 
> information. It is intended solely for the use of the 
> addressee. If you are not the intended recipient, you are 
> strictly prohibited from disclosing, copying, distributing or 
> using any of this information. If you received this 
> communication in error, please contact the sender immediately 
> and destroy the material in its entirety, whether electronic 
> or hard copy. This communication may contain nonpublic 
> personal information about consumers subject to the 
> restrictions of the Gramm-Leach-Bliley Act. You may not 
> directly or indirectly reuse or redisclose such information 
> for any purpose other than to provide the services for which 
> you are receiving the information.
> 
> 127 Public Square, Cleveland, OH 44114
> If you prefer not to receive future e-mail offers for 
> products or services from Key 
> send an e-mail to mailto:dnereque...@key.com with 'No 
> Promotional E-mails' in the 
> SUBJECT line.
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> 
> 
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

[Tom Ambros]

"Complicated legacy mainframe system"?  It's CA-7!  If they think that's 
complicated they probably can't work a Mr. Coffee. 

Thomas Ambros
Operating Systems and Connectivity Engineering
518-436-6433





From:   Ed Gould 
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   06/27/2012 00:44
Subject:'Inexperienced' RBS tech operative's blunder led to 
banking meltdown
Sent by:IBM Mainframe Discussion List 



http://www.theregister.co.uk/2012/06/26/ 
rbs_natwest_ca_technologies_outsourcing/


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

This communication may contain privileged and/or confidential information. It 
is intended solely for the use of the addressee. If you are not the intended 
recipient, you are strictly prohibited from disclosing, copying, distributing 
or using any of this information. If you received this communication in error, 
please contact the sender immediately and destroy the material in its entirety, 
whether electronic or hard copy. This communication may contain nonpublic 
personal information about consumers subject to the restrictions of the 
Gramm-Leach-Bliley Act. You may not directly or indirectly reuse or redisclose 
such information for any purpose other than to provide the services for which 
you are receiving the information.

127 Public Square, Cleveland, OH 44114
If you prefer not to receive future e-mail offers for products or services from 
Key 
send an e-mail to mailto:dnereque...@key.com with 'No Promotional E-mails' in 
the 
SUBJECT line.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN