Re: [IBM External] Re: The Business Case for BatchPipes in the z/OS Base (was: ... Pipes ...)

2021-09-29 Thread Paul Gilmartin 
On Wed, 29 Sep 2021 06:54:09 +, Martin Packer wrote:
>
>Between steps can't be pipes, can be VIO. Between jobs can be pipes, can't
>be VIO.
>
Pipes could work between steps if the PIPE SUBSYS has an implied ELASTIC
stage.  But that's just reinventing VIO.  PIPE SUBSYS must do some buffering
to handle BSAM RECFM=VB.  Or does it require that RECFM and BLKSIZE
or the producer and consumer be identical?

>That second sentence depends on the ability to schedule two jobs (possibly
>originally steps of the same job) alongside each other.
>
Alas, the Initiator won't do the latter for you.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: [IBM External] The Business Case for BatchPipes in the z/OS Base (was: ... Pipes ...)

2021-09-29 Thread kekronbekron 
BTW, are most of these recommendations still valid?

- KB

‐‐‐ Original Message ‐‐‐

On Wednesday, September 29th, 2021 at 7:34 PM, Martin Packer 
 wrote:

> Thank you KB!
>
> You can imagine my disappointment at finding all the links to it at the
>
> IBM Redbooks site to be broken.
>
> Cheers, Martin
>
> Martin Packer
>
> WW z/OS Performance, Capacity and Architecture, IBM Technology Sales
>
> +44-7802-245-584
>
> email: martin_pac...@uk.ibm.com
>
> Twitter / Facebook IDs: MartinPacker
>
> Blog: https://mainframeperformancetopics.com
>
> Mainframe, Performance, Topics Podcast Series (With Marna Walle):
>
> https://anchor.fm/marna-walle
>
> Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA
>
> From: "kekronbekron" 02dee3fcae33-dmarc-requ...@listserv.ua.edu
>
> To: IBM-MAIN@LISTSERV.UA.EDU
>
> Date: 29/09/2021 14:46
>
> Subject: [EXTERNAL] Re: [IBM External] The Business Case for
>
> BatchPipes in the z/OS Base (was: ... Pipes ...)
>
> Sent by: "IBM Mainframe Discussion List" IBM-MAIN@LISTSERV.UA.EDU
>
> Well my friends, I found it for ye.
>
> https://web.archive.org/web/20060224031157/http://www.redbooks.ibm.com/redbooks/pdfs/sg242557.pdf
>
> Saying this to nobody on this thread, but to some kind of people
>
> Now tell me what's important... knowing how to get to stuff or knowing by
>
> memory all the new parmlib members / keywords in zOS 2.5
>
> -   KB
>
> Original Message
>
> On Wednesday, September 29th, 2021 at 6:45 PM, Martin Packer
>
> martin_pac...@uk.ibm.com wrote:
>
> > I think it's gone. Does anyone still have a PDF of it?
> >
> > Thanks, Martin
> >
> > Martin Packer
> >
> > WW z/OS Performance, Capacity and Architecture, IBM Technology Sales
> >
> > +44-7802-245-584
> >
> > email: martin_pac...@uk.ibm.com
> >
> > Twitter / Facebook IDs: MartinPacker
> >
> > Blog:
>
> https://mainframeperformancetopics.com
>
> > Mainframe, Performance, Topics Podcast Series (With Marna Walle):
>
> https://anchor.fm/marna-walle
>
> > Youtube channel:
>
> https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA
>
> > From: "René Jansen" rene.vincent.jan...@gmail.com
> >
> > To: IBM-MAIN@LISTSERV.UA.EDU
> >
> > Date: 29/09/2021 09:12
> >
> > Subject: [EXTERNAL] Re: [IBM External] The Business Case for
> >
> > BatchPipes in the z/OS Base (was: ... Pipes ...)
> >
> > Sent by: "IBM Mainframe Discussion List" IBM-MAIN@LISTSERV.UA.EDU
> >
> > Martin,
> >
> > Do you have that book somewhere still? I spent 15 minutes on google but
> >
> > lots of links lead nowhere.
> >
> > NetRexx does have a rather complete CMS Pipelines implementation (
> >
> > www.netrexx.org <
>
> http://www.netrexx.org/
>
> > > ) and I was wondering what I need to do to also have these ’halfpipes’
> >
> > and other batch interfaces, using JZOS or other.
> >
> > Best regards,
> >
> > René.
> >
> > > On 29 Sep 2021, at 08:54, Martin Packer martin_pac...@uk.ibm.com
> >
> > wrote:
> >
> > > Between steps can't be pipes, can be VIO. Between jobs can be pipes,
> >
> > can't
> >
> > > be VIO.
> > >
> > > That second sentence depends on the ability to schedule two jobs
> >
> > (possibly
> >
> > > originally steps of the same job) alongside each other.
> > >
> > > Fun stuff but / and somewhat complex - which is what inspired me to
> >
> > start
> >
> > > writing what would become SG24-2557 Parallel Sysplex Batch Performance
> >
> > in
> >
> > > late 1990. :-)
> > >
> > > I did a lot of presenting on Pipes to individual customers and
> >
> > conferences
> >
> > > in the 1990s. It would be fun to do it again... :-)
> > >
> > > Cheers, Martin
> > >
> > > Sent from my iPad
> > >
> > > > On 29 Sep 2021, at 05:23, Paul Gilmartin
> > > >
> > > > 000433f07816-dmarc-requ...@listserv.ua.edu wrote:
> > > >
> > > > On Mon, 27 Sep 2021 11:39:17 -0500, Hobart Spitz wrote:
> > > >
> > > > > Intra-JOB pipe - This would be similar to VIO; it doesn't exist
> > > > >
> > > > > otherwise.
> > > >
> > > > > I think it would be a great feature.
> > > >
> > > > Between steps? One might just as well use VIO.
> > > >
> > > > -- gil
> > > >
> > > > For IBM-MAIN subscribe / signoff / archive access instructions,
> > > >
> > > > send email to lists...@listserv.ua.edu with the message: INFO
> >
> > IBM-MAINUnless stated otherwise above:
> >
> > > IBM United Kingdom Limited - Registered in England and Wales with
>
> number
>
> > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
> >
> > 3AU
> >
> > > For IBM-MAIN subscribe / signoff / archive access instructions,
> > >
> > > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> >
> > --
> >
> > For IBM-MAIN subscribe / signoff / archive access instructions,
> >
> > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> >
> > Unless stated otherwise above:
> >
> > IBM United Kingdom Limited - Registered in England and Wales with number
> >
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
>
> 3AU
>
> >

Re: [IBM External] The Business Case for BatchPipes in the z/OS Base (was: ... Pipes ...)

2021-09-29 Thread Martin Packer 
Thank you KB!

You can imagine my disappointment at finding all the links to it at the 
IBM Redbooks site to be broken.

Cheers, Martin

Martin Packer

WW z/OS Performance, Capacity and Architecture, IBM Technology Sales

+44-7802-245-584

email: martin_pac...@uk.ibm.com

Twitter / Facebook IDs: MartinPacker

Blog: https://mainframeperformancetopics.com

Mainframe, Performance, Topics Podcast Series (With Marna Walle): 
https://anchor.fm/marna-walle

Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA



From:   "kekronbekron" <02dee3fcae33-dmarc-requ...@listserv.ua.edu>
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   29/09/2021 14:46
Subject:[EXTERNAL] Re: [IBM External] The Business Case for 
BatchPipes in the z/OS Base (was: ... Pipes ...)
Sent by:"IBM Mainframe Discussion List" 



Well my friends, I found it for ye.

https://web.archive.org/web/20060224031157/http://www.redbooks.ibm.com/redbooks/pdfs/sg242557.pdf
 



*Saying this to nobody on this thread, but to some kind of people*
Now tell me what's important... knowing how to get to stuff or knowing by 
memory all the new parmlib members / keywords in zOS 2.5

- KB

�\�\�\�\�\�\�\ Original Message �\�\�\�\�\�\�\

On Wednesday, September 29th, 2021 at 6:45 PM, Martin Packer 
 wrote:

> I think it's gone. Does anyone still have a PDF of it?
>
> Thanks, Martin
>
> Martin Packer
>
> WW z/OS Performance, Capacity and Architecture, IBM Technology Sales
>
> +44-7802-245-584
>
> email: martin_pac...@uk.ibm.com
>
> Twitter / Facebook IDs: MartinPacker
>
> Blog: 
https://mainframeperformancetopics.com 

>
> Mainframe, Performance, Topics Podcast Series (With Marna Walle):
>
> 
https://anchor.fm/marna-walle 

>
> Youtube channel: 
https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA 

>
> From: "René Jansen" rene.vincent.jan...@gmail.com
>
> To: IBM-MAIN@LISTSERV.UA.EDU
>
> Date: 29/09/2021 09:12
>
> Subject: [EXTERNAL] Re: [IBM External] The Business Case for
>
> BatchPipes in the z/OS Base (was: ... Pipes ...)
>
> Sent by: "IBM Mainframe Discussion List" IBM-MAIN@LISTSERV.UA.EDU
>
> Martin,
>
> Do you have that book somewhere still? I spent 15 minutes on google but
>
> lots of links lead nowhere.
>
> NetRexx does have a rather complete CMS Pipelines implementation (
>
> www.netrexx.org <
>
> 
http://www.netrexx.org/ 

>
> > ) and I was wondering what I need to do to also have these ’halfpipes’
>
> and other batch interfaces, using JZOS or other.
>
> Best regards,
>
> René.
>
> > On 29 Sep 2021, at 08:54, Martin Packer martin_pac...@uk.ibm.com
>
> wrote:
>
> > Between steps can't be pipes, can be VIO. Between jobs can be pipes,
>
> can't
>
> > be VIO.
> >
> > That second sentence depends on the ability to schedule two jobs
>
> (possibly
>
> > originally steps of the same job) alongside each other.
> >
> > Fun stuff but / and somewhat complex - which is what inspired me to
>
> start
>
> > writing what would become SG24-2557 Parallel Sysplex Batch Performance
>
> in
>
> > late 1990. :-)
> >
> > I did a lot of presenting on Pipes to individual customers and
>
> conferences
>
> > in the 1990s. It would be fun to do it again... :-)
> >
> > Cheers, Martin
> >
> > Sent from my iPad
> >
> > > On 29 Sep 2021, at 05:23, Paul Gilmartin
> > >
> > > 000433f07816-dmarc-requ...@listserv.ua.edu wrote:
> > >
> > > On Mon, 27 Sep 2021 11:39:17 -0500, Hobart Spitz wrote:
> > >
> > > > Intra-JOB pipe - This would be similar to VIO; it doesn't exist
> > > >
> > > > otherwise.
> > >
> > > > I think it would be a great feature.
> > >
> > > Between steps? One might just as well use VIO.
> > >
> > > -- gil
> > >
> > > For IBM-MAIN subscribe / signoff / archive access instructions,
> > >
> > > send email to lists...@listserv.ua.edu with the message: INFO
>
> IBM-MAINUnless stated otherwise above:
>
> > IBM United Kingdom Limited - Registered in England and Wales with 
number
>
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
>
> 3AU
>
> > For IBM-MAIN subscribe / signoff / archive access instructions,
> >
> > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>
> --
>
> For IBM-MAIN subscribe / signoff / archive access instructions,
>
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>
> Unless stated otherwise above:
>
> IBM United Kingdom Limited - Registered in England and Wales with number
>
> 741598.
>
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 
3AU
>
>
> 

>
> For IBM-MAIN subscribe / signoff / archive access instructions,
>
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: [IBM External] The Business Case for BatchPipes in the z/OS Base (was: ... Pipes ...)

2021-09-29 Thread Ron Wells 
Ahh basicsI remember

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
kekronbekron
Sent: Wednesday, September 29, 2021 8:46 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: [IBM External] The Business Case for BatchPipes in the z/OS Base 
(was: ... Pipes ...)

** EXTERNAL EMAIL - USE CAUTION **


Well my friends, I found it for ye.

https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fweb.archive.org%2Fweb%2F20060224031157%2Fhttp%3A%2F%2Fwww.redbooks.ibm.com%2Fredbooks%2Fpdfs%2Fsg242557.pdfdata=04%7C01%7CRon.Wells%40OMF.COM%7C5d18d65d03ac481b280408d9834f7854%7C57c0053cb5f84a1e8bb6e8afa09f3b82%7C0%7C0%7C63768520084735%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=QrpAr3pkkI4EqsiRBLS8xMBCqEn59a3xvdfeO6woEGk%3Dreserved=0


*Saying this to nobody on this thread, but to some kind of people* Now tell me 
what's important... knowing how to get to stuff or knowing by memory all the 
new parmlib members / keywords in zOS 2.5

- KB

‐‐‐ Original Message ‐‐‐

On Wednesday, September 29th, 2021 at 6:45 PM, Martin Packer 
 wrote:

> I think it's gone. Does anyone still have a PDF of it?
>
> Thanks, Martin
>
> Martin Packer
>
> WW z/OS Performance, Capacity and Architecture, IBM Technology Sales
>
> +44-7802-245-584
>
> email: martin_pac...@uk.ibm.com
>
> Twitter / Facebook IDs: MartinPacker
>
> Blog:
> https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmain
> frameperformancetopics.com%2Fdata=04%7C01%7CRon.Wells%40OMF.COM%7
> C5d18d65d03ac481b280408d9834f7854%7C57c0053cb5f84a1e8bb6e8afa09f3b82%7
> C0%7C0%7C637685200847310001%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMD
> AiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=nYYf
> Xc%2BO%2F4BhKOG9G5yNW5OgSVXUpPSDg1eDT1tBvx0%3Dreserved=0
>
> Mainframe, Performance, Topics Podcast Series (With Marna Walle):
>
> https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fanch
> or.fm%2Fmarna-walledata=04%7C01%7CRon.Wells%40OMF.COM%7C5d18d65d0
> 3ac481b280408d9834f7854%7C57c0053cb5f84a1e8bb6e8afa09f3b82%7C0%7C0%7C6
> 37685200847310001%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=6laUrA7rfLQ9FG
> Ion0vaCk8qODQGTGLBs8kcXaNsOfY%3Dreserved=0
>
> Youtube channel:
> https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> youtube.com%2Fchannel%2FUCu_65HaYgksbF6Q8SQ4oOvAdata=04%7C01%7CRo
> n.Wells%40OMF.COM%7C5d18d65d03ac481b280408d9834f7854%7C57c0053cb5f84a1
> e8bb6e8afa09f3b82%7C0%7C0%7C637685200847310001%7CUnknown%7CTWFpbGZsb3d
> 8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
> 3000sdata=prb%2FQMce629xpliec12VBsH378IG7cbKVc66rA6WW7g%3Dre
> served=0
>
> From: "René Jansen" rene.vincent.jan...@gmail.com
>
> To: IBM-MAIN@LISTSERV.UA.EDU
>
> Date: 29/09/2021 09:12
>
> Subject: [EXTERNAL] Re: [IBM External] The Business Case for
>
> BatchPipes in the z/OS Base (was: ... Pipes ...)
>
> Sent by: "IBM Mainframe Discussion List" IBM-MAIN@LISTSERV.UA.EDU
>
> Martin,
>
> Do you have that book somewhere still? I spent 15 minutes on google
> but
>
> lots of links lead nowhere.
>
> NetRexx does have a rather complete CMS Pipelines implementation (
>
> https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.n
> etrexx.org%2Fdata=04%7C01%7CRon.Wells%40OMF.COM%7C5d18d65d03ac481
> b280408d9834f7854%7C57c0053cb5f84a1e8bb6e8afa09f3b82%7C0%7C0%7C6376852
> 00847310001%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMz
> IiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=7G0d%2BLopX5dw7x2O6f
> Imcn83jLCAoRLLJVM0aObRkrg%3Dreserved=0 <
>
> https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.n
> etrexx.org%2Fdata=04%7C01%7CRon.Wells%40OMF.COM%7C5d18d65d03ac481
> b280408d9834f7854%7C57c0053cb5f84a1e8bb6e8afa09f3b82%7C0%7C0%7C6376852
> 00847310001%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMz
> IiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=7G0d%2BLopX5dw7x2O6f
> Imcn83jLCAoRLLJVM0aObRkrg%3Dreserved=0
>
> > ) and I was wondering what I need to do to also have these ’halfpipes’
>
> and other batch interfaces, using JZOS or other.
>
> Best regards,
>
> René.
>
> > On 29 Sep 2021, at 08:54, Martin Packer martin_pac...@uk.ibm.com
>
> wrote:
>
> > Between steps can't be pipes, can be VIO. Between jobs can be pipes,
>
> can't
>
> > be VIO.
> >
> > That second sentence depends on the ability to schedule two jobs
>
> (possibly
>
> > originally steps of the same job) alongside each other.
> >
> > Fun stuff but / and somewhat complex - which is what inspired me to
>
> start
>
> > writing what would become SG24-2557 Parallel Sysplex Batch
> > Performance
>
> in
>
> > late 1990. :-)
> >
> > I did a lot of presenting on Pipes to individual customers and
>
> conferences
>
> > in the 1990s. It would be fun to do it again... :-)
> >
> > Cheers, Martin
> >
> > Sent from my iPad
> >
> > > On 29 Sep 2021, at 05:23, Paul Gilmartin
> > >
> > >

Re: [IBM External] The Business Case for BatchPipes in the z/OS Base (was: ... Pipes ...)

2021-09-29 Thread kekronbekron 
Well my friends, I found it for ye.

https://web.archive.org/web/20060224031157/http://www.redbooks.ibm.com/redbooks/pdfs/sg242557.pdf


*Saying this to nobody on this thread, but to some kind of people*
Now tell me what's important... knowing how to get to stuff or knowing by 
memory all the new parmlib members / keywords in zOS 2.5

- KB

‐‐‐ Original Message ‐‐‐

On Wednesday, September 29th, 2021 at 6:45 PM, Martin Packer 
 wrote:

> I think it's gone. Does anyone still have a PDF of it?
>
> Thanks, Martin
>
> Martin Packer
>
> WW z/OS Performance, Capacity and Architecture, IBM Technology Sales
>
> +44-7802-245-584
>
> email: martin_pac...@uk.ibm.com
>
> Twitter / Facebook IDs: MartinPacker
>
> Blog: https://mainframeperformancetopics.com
>
> Mainframe, Performance, Topics Podcast Series (With Marna Walle):
>
> https://anchor.fm/marna-walle
>
> Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA
>
> From: "René Jansen" rene.vincent.jan...@gmail.com
>
> To: IBM-MAIN@LISTSERV.UA.EDU
>
> Date: 29/09/2021 09:12
>
> Subject: [EXTERNAL] Re: [IBM External] The Business Case for
>
> BatchPipes in the z/OS Base (was: ... Pipes ...)
>
> Sent by: "IBM Mainframe Discussion List" IBM-MAIN@LISTSERV.UA.EDU
>
> Martin,
>
> Do you have that book somewhere still? I spent 15 minutes on google but
>
> lots of links lead nowhere.
>
> NetRexx does have a rather complete CMS Pipelines implementation (
>
> www.netrexx.org <
>
> http://www.netrexx.org/
>
> > ) and I was wondering what I need to do to also have these ’halfpipes’
>
> and other batch interfaces, using JZOS or other.
>
> Best regards,
>
> René.
>
> > On 29 Sep 2021, at 08:54, Martin Packer martin_pac...@uk.ibm.com
>
> wrote:
>
> > Between steps can't be pipes, can be VIO. Between jobs can be pipes,
>
> can't
>
> > be VIO.
> >
> > That second sentence depends on the ability to schedule two jobs
>
> (possibly
>
> > originally steps of the same job) alongside each other.
> >
> > Fun stuff but / and somewhat complex - which is what inspired me to
>
> start
>
> > writing what would become SG24-2557 Parallel Sysplex Batch Performance
>
> in
>
> > late 1990. :-)
> >
> > I did a lot of presenting on Pipes to individual customers and
>
> conferences
>
> > in the 1990s. It would be fun to do it again... :-)
> >
> > Cheers, Martin
> >
> > Sent from my iPad
> >
> > > On 29 Sep 2021, at 05:23, Paul Gilmartin
> > >
> > > 000433f07816-dmarc-requ...@listserv.ua.edu wrote:
> > >
> > > On Mon, 27 Sep 2021 11:39:17 -0500, Hobart Spitz wrote:
> > >
> > > > Intra-JOB pipe - This would be similar to VIO; it doesn't exist
> > > >
> > > > otherwise.
> > >
> > > > I think it would be a great feature.
> > >
> > > Between steps? One might just as well use VIO.
> > >
> > > -- gil
> > >
> > > For IBM-MAIN subscribe / signoff / archive access instructions,
> > >
> > > send email to lists...@listserv.ua.edu with the message: INFO
>
> IBM-MAINUnless stated otherwise above:
>
> > IBM United Kingdom Limited - Registered in England and Wales with number
>
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
>
> 3AU
>
> > For IBM-MAIN subscribe / signoff / archive access instructions,
> >
> > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>
> --
>
> For IBM-MAIN subscribe / signoff / archive access instructions,
>
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>
> Unless stated otherwise above:
>
> IBM United Kingdom Limited - Registered in England and Wales with number
>
> 741598.
>
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
>
> 
>
> For IBM-MAIN subscribe / signoff / archive access instructions,
>
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: [IBM External] The Business Case for BatchPipes in the z/OS Base (was: ... Pipes ...)

2021-09-29 Thread Martin Packer 
I think it's gone. Does anyone still have a PDF of it?

Thanks, Martin

Martin Packer

WW z/OS Performance, Capacity and Architecture, IBM Technology Sales

+44-7802-245-584

email: martin_pac...@uk.ibm.com

Twitter / Facebook IDs: MartinPacker

Blog: https://mainframeperformancetopics.com

Mainframe, Performance, Topics Podcast Series (With Marna Walle): 
https://anchor.fm/marna-walle

Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA



From:   "René Jansen" 
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   29/09/2021 09:12
Subject:[EXTERNAL] Re: [IBM External] The Business Case for 
BatchPipes in the z/OS Base (was: ... Pipes ...)
Sent by:"IBM Mainframe Discussion List" 



Martin,

Do you have that book somewhere still? I spent 15 minutes on google but 
lots of links lead nowhere.

NetRexx does have a rather complete CMS Pipelines implementation (
www.netrexx.org <
http://www.netrexx.org/ 
>) and I was wondering what I need to do to also have these ’halfpipes’ 
and other batch interfaces, using JZOS or other.

Best regards,

René. 


> On 29 Sep 2021, at 08:54, Martin Packer  
wrote:
> 
> 
> 
> Between steps can't be pipes, can be VIO. Between jobs can be pipes, 
can't
> be VIO.
> 
> That second sentence depends on the ability to schedule two jobs 
(possibly
> originally steps of the same job) alongside each other.
> 
> Fun stuff but / and somewhat complex - which is what inspired me to 
start
> writing what would become SG24-2557 Parallel Sysplex Batch Performance 
in
> late 1990. :-)
> 
> I did a lot of presenting on Pipes to individual customers and 
conferences
> in the 1990s. It would be fun to do it again... :-)
> 
> Cheers, Martin
> 
> Sent from my iPad
> 
>> On 29 Sep 2021, at 05:23, Paul Gilmartin
> <000433f07816-dmarc-requ...@listserv.ua.edu> wrote:
>> 
>> On Mon, 27 Sep 2021 11:39:17 -0500, Hobart Spitz wrote:
>>> 
>>> Intra-JOB pipe - This would be similar to VIO; it doesn't exist
> otherwise.
>>> I think it would be a great feature.
>>> 
>> Between steps?  One might just as well use VIO.
>> 
>> -- gil
>> 
>> --
>> For IBM-MAIN subscribe / signoff / archive access instructions,
>> send email to lists...@listserv.ua.edu with the message: INFO 
IBM-MAINUnless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 
3AU
> 
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN



Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: [IBM External] The Business Case for BatchPipes in the z/OS Base (was: ... Pipes ...)

2021-09-29 Thread René Jansen 
Martin,

Do you have that book somewhere still? I spent 15 minutes on google but lots of 
links lead nowhere.

NetRexx does have a rather complete CMS Pipelines implementation 
(www.netrexx.org ) and I was wondering what I need to 
do to also have these ’halfpipes’ and other batch interfaces, using JZOS or 
other.

Best regards,

René. 


> On 29 Sep 2021, at 08:54, Martin Packer  wrote:
> 
> 
> 
> Between steps can't be pipes, can be VIO. Between jobs can be pipes, can't
> be VIO.
> 
> That second sentence depends on the ability to schedule two jobs (possibly
> originally steps of the same job) alongside each other.
> 
> Fun stuff but / and somewhat complex - which is what inspired me to start
> writing what would become SG24-2557 Parallel Sysplex Batch Performance in
> late 1990. :-)
> 
> I did a lot of presenting on Pipes to individual customers and conferences
> in the 1990s. It would be fun to do it again... :-)
> 
> Cheers, Martin
> 
> Sent from my iPad
> 
>> On 29 Sep 2021, at 05:23, Paul Gilmartin
> <000433f07816-dmarc-requ...@listserv.ua.edu> wrote:
>> 
>> On Mon, 27 Sep 2021 11:39:17 -0500, Hobart Spitz wrote:
>>> 
>>> Intra-JOB pipe - This would be similar to VIO; it doesn't exist
> otherwise.
>>> I think it would be a great feature.
>>> 
>> Between steps?  One might just as well use VIO.
>> 
>> -- gil
>> 
>> --
>> For IBM-MAIN subscribe / signoff / archive access instructions,
>> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAINUnless 
>> stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 
> 741598. 
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
> 
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: [IBM External] Re: The Business Case for BatchPipes in the z/OS Base (was: ... Pipes ...)

2021-09-29 Thread Martin Packer 


Between steps can't be pipes, can be VIO. Between jobs can be pipes, can't
be VIO.

That second sentence depends on the ability to schedule two jobs (possibly
originally steps of the same job) alongside each other.

Fun stuff but / and somewhat complex - which is what inspired me to start
writing what would become SG24-2557 Parallel Sysplex Batch Performance in
late 1990. :-)

I did a lot of presenting on Pipes to individual customers and conferences
in the 1990s. It would be fun to do it again... :-)

Cheers, Martin

Sent from my iPad

> On 29 Sep 2021, at 05:23, Paul Gilmartin
<000433f07816-dmarc-requ...@listserv.ua.edu> wrote:
>
> On Mon, 27 Sep 2021 11:39:17 -0500, Hobart Spitz wrote:
>>
>> Intra-JOB pipe - This would be similar to VIO; it doesn't exist
otherwise.
>> I think it would be a great feature.
>>
> Between steps?  One might just as well use VIO.
>
> -- gil
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAINUnless 
> stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: The Business Case for BatchPipes in the z/OS Base (was: ... Pipes ...)

2021-09-28 Thread Paul Gilmartin 
On Mon, 27 Sep 2021 11:39:17 -0500, Hobart Spitz wrote:
>
>Intra-JOB pipe - This would be similar to VIO; it doesn't exist otherwise.
>I think it would be a great feature. 
>
Between steps?  One might just as well use VIO.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: The Business Case for BatchPipes in the z/OS Base (was: ... Pipes ...)

2021-09-28 Thread David Crayford 
Thanks for the clarity. So BatchPipes are the plumbing to implement 
streaming almost like the JCL equivalent of the Apache Kafka Streams API?


On 28/09/2021 4:00 pm, Martin Packer wrote:

I think you have to remember that BatchPipes/MVS' origin story is
connecting existing batch job steps. I wouldn't want customers to have to
re-write e.g. in java.

One advantage is that it's easier to rework a pipeline - whether as a
fitting or between batch jobs - than rework some java code.

As someone who first proselytised Pipes in 1992 and first wrote about it
in a Redbook in 1997 (and wrote about it again in 2011 and 2013) you can
consider me a fan. Note: I don't have an enormous amount of influence on
those that make decisions about either IBM flavour of pipes. I do
recognise it's not as simple as "set the code free". There is the cost of
bringing it to market and supporting it. The latter in particular would be
much more costly than it's ever been before - if it were built in.

So, I hope we can make both Pipelines and BatchPipes/MVS available as part
of z/OS. I'd love to be talking about them again and supporting customers
using them. But I'm just a field guy.

Cheers, Martin

Martin Packer

WW z/OS Performance, Capacity and Architecture, IBM Technology Sales

+44-7802-245-584

email: martin_pac...@uk.ibm.com

Twitter / Facebook IDs: MartinPacker

Blog: https://mainframeperformancetopics.com

Mainframe, Performance, Topics Podcast Series (With Marna Walle):
https://anchor.fm/marna-walle

Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA



From:   "David Crayford" 
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   28/09/2021 08:17
Subject:[EXTERNAL] Re: The Business Case for BatchPipes in the
z/OS Base (was: ... Pipes ...)
Sent by:"IBM Mainframe Discussion List" 



This first question I would ask is does IBM actually "own" BatchPipes or
did the flog it off to an ISV like most of their other software? If it's
the latter then they will be in no position to make it freely available.

Secondly, rather than pine for something that isn't available why not
just switch technologies and use a language that supports functional
programming features that are similar to pipes. Even Java has supported
functional programming since Java 8 came with streams in 2014.

https://stackify.com/streams-guide-java-8/



On 28/09/2021 12:39 am, Hobart Spitz wrote:

Gil wrote:

On Mon, 27 Sep 2021 07:26:51 -0500, Hobart Spitz wrote:

I'm going to pivot here.  I'm putting my support behind putting

BatchPipes

in the z/OS base (rather than just Pipes).  If you agree, please
write/support such a requirement and/or educate your management to get
interested.  BatchPipes includes BatchPipesWorks, a not so current,

but

still highly useful, version of TSO Pipelines.


Is an RFE for an update required?  Conway's Law.

Great question.  I guess it would depend on whether the were a lot of
current BatchPipes customers needed the missing builtin stages and/o
fix(es) or whether it was more important to get BatchPipes in the z/OS
base.  It might also depend on how much additional work would be

required

to get a more current version of Pipes into BatchPipes, and whether the
staff, skills, and funding were available.  IMHO, the update would delay
the benefits of BatchPipes in the base (especially global warming
mitigation), and not be important to current BatchPipes customers.  In

my

estimation, BatchPipesWorks has 90% of the needed stages and 99% of the
fix(es) as would be in an up to date version of TSO Pipelines.  I find

the

IBM decision making process obscure, so these may not be the

considerations

that affect the final decision.

AFAIK, there is only one person now working on Pipes, so I must be

missing

something in applying Conway's law.


Does BatchPipes support connecting two Classic modules with an

intervening

small Pipeline filter?  How?  Is a coordinated third job needed?

Let me clarify some terms, answer you questions in the process, and

clarify

my previous post:


Inter-JOB pipe - This is a ppe that let's two JOB pass records from one

to

the next thru memory.  The BatchPipes subsystem(s) is/are required.
AFAIK:  There are no obvious limitations to the topology.  You can have
multiple JOB connected in a single "stream", split and/or join streams,

or

even have loop(s).  In this respect, it is similar to the PIPE command,
except that the record flow with split and rejoined streams may not be
predictable between JOBs.

Half Pipe Fitting or Intra-STEP pipe - This is a series of Pipes stages
that operate between a program in a step and the storage medium.

Intra-JOB pipe - This would be similar to VIO; it doesn't exist

otherwise.

I think it would be a great feature.  Hence I incorrectly used the term

to

refer to Half Pipe Fittings in a generic sense.  Apologies for any
confusion.


AFAIK, and if I understood your question, connecting two classic

programs

in the same step with a series of

Re: The Business Case for BatchPipes in the z/OS Base (was: ... Pipes ...)

2021-09-28 Thread Martin Packer 
I think you have to remember that BatchPipes/MVS' origin story is 
connecting existing batch job steps. I wouldn't want customers to have to 
re-write e.g. in java.

One advantage is that it's easier to rework a pipeline - whether as a 
fitting or between batch jobs - than rework some java code.

As someone who first proselytised Pipes in 1992 and first wrote about it 
in a Redbook in 1997 (and wrote about it again in 2011 and 2013) you can 
consider me a fan. Note: I don't have an enormous amount of influence on 
those that make decisions about either IBM flavour of pipes. I do 
recognise it's not as simple as "set the code free". There is the cost of 
bringing it to market and supporting it. The latter in particular would be 
much more costly than it's ever been before - if it were built in.

So, I hope we can make both Pipelines and BatchPipes/MVS available as part 
of z/OS. I'd love to be talking about them again and supporting customers 
using them. But I'm just a field guy.

Cheers, Martin

Martin Packer

WW z/OS Performance, Capacity and Architecture, IBM Technology Sales

+44-7802-245-584

email: martin_pac...@uk.ibm.com

Twitter / Facebook IDs: MartinPacker

Blog: https://mainframeperformancetopics.com

Mainframe, Performance, Topics Podcast Series (With Marna Walle): 
https://anchor.fm/marna-walle

Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA



From:   "David Crayford" 
To: IBM-MAIN@LISTSERV.UA.EDU
Date:   28/09/2021 08:17
Subject:[EXTERNAL] Re: The Business Case for BatchPipes in the 
z/OS Base (was: ... Pipes ...)
Sent by:"IBM Mainframe Discussion List" 



This first question I would ask is does IBM actually "own" BatchPipes or 
did the flog it off to an ISV like most of their other software? If it's 
the latter then they will be in no position to make it freely available.

Secondly, rather than pine for something that isn't available why not 
just switch technologies and use a language that supports functional 
programming features that are similar to pipes. Even Java has supported 
functional programming since Java 8 came with streams in 2014.

https://stackify.com/streams-guide-java-8/ 



On 28/09/2021 12:39 am, Hobart Spitz wrote:
> Gil wrote:
>> On Mon, 27 Sep 2021 07:26:51 -0500, Hobart Spitz wrote:
>>> I'm going to pivot here.  I'm putting my support behind putting 
BatchPipes
>>> in the z/OS base (rather than just Pipes).  If you agree, please
>>> write/support such a requirement and/or educate your management to get
>>> interested.  BatchPipes includes BatchPipesWorks, a not so current, 
but
>>> still highly useful, version of TSO Pipelines.
>>>
>> Is an RFE for an update required?  Conway's Law.
> Great question.  I guess it would depend on whether the were a lot of
> current BatchPipes customers needed the missing builtin stages and/o
> fix(es) or whether it was more important to get BatchPipes in the z/OS
> base.  It might also depend on how much additional work would be 
required
> to get a more current version of Pipes into BatchPipes, and whether the
> staff, skills, and funding were available.  IMHO, the update would delay
> the benefits of BatchPipes in the base (especially global warming
> mitigation), and not be important to current BatchPipes customers.  In 
my
> estimation, BatchPipesWorks has 90% of the needed stages and 99% of the
> fix(es) as would be in an up to date version of TSO Pipelines.  I find 
the
> IBM decision making process obscure, so these may not be the 
considerations
> that affect the final decision.
>
> AFAIK, there is only one person now working on Pipes, so I must be 
missing
> something in applying Conway's law.
>
>> Does BatchPipes support connecting two Classic modules with an 
intervening
>> small Pipeline filter?  How?  Is a coordinated third job needed?
> Let me clarify some terms, answer you questions in the process, and 
clarify
> my previous post:
>
>
> Inter-JOB pipe - This is a ppe that let's two JOB pass records from one 
to
> the next thru memory.  The BatchPipes subsystem(s) is/are required.
> AFAIK:  There are no obvious limitations to the topology.  You can have
> multiple JOB connected in a single "stream", split and/or join streams, 
or
> even have loop(s).  In this respect, it is similar to the PIPE command,
> except that the record flow with split and rejoined streams may not be
> predictable between JOBs.
>
> Half Pipe Fitting or Intra-STEP pipe - This is a series of Pipes stages
> that operate between a program in a step and the storage medium.
>
> Intra-JOB pipe - This would be similar to VIO; it doesn't exist 
otherwise.
> I think it would be a great feature.  Hence I incorrectly used the term 
to
> refer to Half Pipe Fittings in a generic sense.  Apologies for any
> confusion.
>
>
> AFAIK, and if I understood your question, connecting two classic 
programs
> in the same step with a series of Pipes stages may not be  possible now, 
at
> least without resorting to Assembler.

Re: The Business Case for BatchPipes in the z/OS Base (was: ... Pipes ...)

2021-09-28 Thread David Crayford 
This first question I would ask is does IBM actually "own" BatchPipes or 
did the flog it off to an ISV like most of their other software? If it's 
the latter then they will be in no position to make it freely available.


Secondly, rather than pine for something that isn't available why not 
just switch technologies and use a language that supports functional 
programming features that are similar to pipes. Even Java has supported 
functional programming since Java 8 came with streams in 2014.


https://stackify.com/streams-guide-java-8/


On 28/09/2021 12:39 am, Hobart Spitz wrote:

Gil wrote:

On Mon, 27 Sep 2021 07:26:51 -0500, Hobart Spitz wrote:

I'm going to pivot here.  I'm putting my support behind putting BatchPipes
in the z/OS base (rather than just Pipes).  If you agree, please
write/support such a requirement and/or educate your management to get
interested.  BatchPipes includes BatchPipesWorks, a not so current, but
still highly useful, version of TSO Pipelines.


Is an RFE for an update required?  Conway's Law.

Great question.  I guess it would depend on whether the were a lot of
current BatchPipes customers needed the missing builtin stages and/o
fix(es) or whether it was more important to get BatchPipes in the z/OS
base.  It might also depend on how much additional work would be required
to get a more current version of Pipes into BatchPipes, and whether the
staff, skills, and funding were available.  IMHO, the update would delay
the benefits of BatchPipes in the base (especially global warming
mitigation), and not be important to current BatchPipes customers.  In my
estimation, BatchPipesWorks has 90% of the needed stages and 99% of the
fix(es) as would be in an up to date version of TSO Pipelines.  I find the
IBM decision making process obscure, so these may not be the considerations
that affect the final decision.

AFAIK, there is only one person now working on Pipes, so I must be missing
something in applying Conway's law.


Does BatchPipes support connecting two Classic modules with an intervening
small Pipeline filter?  How?  Is a coordinated third job needed?

Let me clarify some terms, answer you questions in the process, and clarify
my previous post:


Inter-JOB pipe - This is a ppe that let's two JOB pass records from one to
the next thru memory.  The BatchPipes subsystem(s) is/are required.
AFAIK:  There are no obvious limitations to the topology.  You can have
multiple JOB connected in a single "stream", split and/or join streams, or
even have loop(s).  In this respect, it is similar to the PIPE command,
except that the record flow with split and rejoined streams may not be
predictable between JOBs.

Half Pipe Fitting or Intra-STEP pipe - This is a series of Pipes stages
that operate between a program in a step and the storage medium.

Intra-JOB pipe - This would be similar to VIO; it doesn't exist otherwise.
I think it would be a great feature.  Hence I incorrectly used the term to
refer to Half Pipe Fittings in a generic sense.  Apologies for any
confusion.


AFAIK, and if I understood your question, connecting two classic programs
in the same step with a series of Pipes stages may not be  possible now, at
least without resorting to Assembler.  I think the main reason is the
potential for DD name conflict, both in the TIOT (DD name table) and in
JCL.  Most COBOL programs require SYSOUT, so the output could be
intermixed.  Much less tolerable, I suspect, would be a common DD (e.g.
OUTPUT) conflict.  Putting a record that doesn't belong between two records
that must be consecutive would probably be a disaster.

That said, it may be possible for a REXX filter to start two COBOL (e.g.)
programs as subtasks (address attach ... , etc.) and feed and receive their
records via an Assembler(?) Pipe Fitting.  You would have to resolve any DD
name conflicts.  I don't think anyone has contemplated the capability for
creating multiple TIOTs in a single step, let alone how to reference them
in JCL.  I think such things have been done under CMS using Assembler.

Also:

pipe - a generic connection, in memory, between two processes or tasks.

Pipes - the TSO/CMS Pipelines software, singular.

Pipe - a specific instance of Pipes.

PIPE - the command that runs Pipes.


OREXXMan
Would you rather pass data in move mode (*nix piping) or locate mode
(Pipes) or via disk (JCL)?  Why do you think you rarely see *nix commands
with more than a dozen filters, while Pipelines specifications are commonly
over 100s of stages, and 1000s of stages are not uncommon.
IBM has been looking for an HLL for program products; REXX is that language.


On Mon, Sep 27, 2021 at 9:10 AM Paul Gilmartin <
000433f07816-dmarc-requ...@listserv.ua.edu> wrote:


On Mon, 27 Sep 2021 07:26:51 -0500, Hobart Spitz wrote:


I'm going to pivot here.  I'm putting my support behind putting BatchPipes
in the z/OS base (rather than just Pipes).  If you agree, please
write/support such a requirement and/or educate your management to

Re: The Business Case for BatchPipes in the z/OS Base (was: ... Pipes ...)

2021-09-27 Thread Hobart Spitz 
Gil wrote:
>On Mon, 27 Sep 2021 07:26:51 -0500, Hobart Spitz wrote:
>>I'm going to pivot here.  I'm putting my support behind putting BatchPipes
>>in the z/OS base (rather than just Pipes).  If you agree, please
>>write/support such a requirement and/or educate your management to get
>>interested.  BatchPipes includes BatchPipesWorks, a not so current, but
>>still highly useful, version of TSO Pipelines.
>>
>Is an RFE for an update required?  Conway's Law.

Great question.  I guess it would depend on whether the were a lot of
current BatchPipes customers needed the missing builtin stages and/o
fix(es) or whether it was more important to get BatchPipes in the z/OS
base.  It might also depend on how much additional work would be required
to get a more current version of Pipes into BatchPipes, and whether the
staff, skills, and funding were available.  IMHO, the update would delay
the benefits of BatchPipes in the base (especially global warming
mitigation), and not be important to current BatchPipes customers.  In my
estimation, BatchPipesWorks has 90% of the needed stages and 99% of the
fix(es) as would be in an up to date version of TSO Pipelines.  I find the
IBM decision making process obscure, so these may not be the considerations
that affect the final decision.

AFAIK, there is only one person now working on Pipes, so I must be missing
something in applying Conway's law.

> Does BatchPipes support connecting two Classic modules with an intervening
> small Pipeline filter?  How?  Is a coordinated third job needed?

Let me clarify some terms, answer you questions in the process, and clarify
my previous post:


Inter-JOB pipe - This is a ppe that let's two JOB pass records from one to
the next thru memory.  The BatchPipes subsystem(s) is/are required.
AFAIK:  There are no obvious limitations to the topology.  You can have
multiple JOB connected in a single "stream", split and/or join streams, or
even have loop(s).  In this respect, it is similar to the PIPE command,
except that the record flow with split and rejoined streams may not be
predictable between JOBs.

Half Pipe Fitting or Intra-STEP pipe - This is a series of Pipes stages
that operate between a program in a step and the storage medium.

Intra-JOB pipe - This would be similar to VIO; it doesn't exist otherwise.
I think it would be a great feature.  Hence I incorrectly used the term to
refer to Half Pipe Fittings in a generic sense.  Apologies for any
confusion.


AFAIK, and if I understood your question, connecting two classic programs
in the same step with a series of Pipes stages may not be  possible now, at
least without resorting to Assembler.  I think the main reason is the
potential for DD name conflict, both in the TIOT (DD name table) and in
JCL.  Most COBOL programs require SYSOUT, so the output could be
intermixed.  Much less tolerable, I suspect, would be a common DD (e.g.
OUTPUT) conflict.  Putting a record that doesn't belong between two records
that must be consecutive would probably be a disaster.

That said, it may be possible for a REXX filter to start two COBOL (e.g.)
programs as subtasks (address attach ... , etc.) and feed and receive their
records via an Assembler(?) Pipe Fitting.  You would have to resolve any DD
name conflicts.  I don't think anyone has contemplated the capability for
creating multiple TIOTs in a single step, let alone how to reference them
in JCL.  I think such things have been done under CMS using Assembler.

Also:

pipe - a generic connection, in memory, between two processes or tasks.

Pipes - the TSO/CMS Pipelines software, singular.

Pipe - a specific instance of Pipes.

PIPE - the command that runs Pipes.


OREXXMan
Would you rather pass data in move mode (*nix piping) or locate mode
(Pipes) or via disk (JCL)?  Why do you think you rarely see *nix commands
with more than a dozen filters, while Pipelines specifications are commonly
over 100s of stages, and 1000s of stages are not uncommon.
IBM has been looking for an HLL for program products; REXX is that language.


On Mon, Sep 27, 2021 at 9:10 AM Paul Gilmartin <
000433f07816-dmarc-requ...@listserv.ua.edu> wrote:

> On Mon, 27 Sep 2021 07:26:51 -0500, Hobart Spitz wrote:
>
> >I'm going to pivot here.  I'm putting my support behind putting BatchPipes
> >in the z/OS base (rather than just Pipes).  If you agree, please
> >write/support such a requirement and/or educate your management to get
> >interested.  BatchPipes includes BatchPipesWorks, a not so current, but
> >still highly useful, version of TSO Pipelines.
> >
> Is an RFE for an update rrequired?  Conway's Law.
>
> Does BatchPipes support connecting two Classic modules with an intervening
> small Pipeline filter?  How?  Is a coordinated third job needed?
>
> >The reasons are:  [Snip!  See the archive.]
>
> -- gil
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to

Re: The Business Case for BatchPipes in the z/OS Base (was: ... Pipes ...)

2021-09-27 Thread Paul Gilmartin 
On Mon, 27 Sep 2021 07:26:51 -0500, Hobart Spitz wrote:

>I'm going to pivot here.  I'm putting my support behind putting BatchPipes
>in the z/OS base (rather than just Pipes).  If you agree, please
>write/support such a requirement and/or educate your management to get
>interested.  BatchPipes includes BatchPipesWorks, a not so current, but
>still highly useful, version of TSO Pipelines.
>
Is an RFE for an update rrequired?  Conway's Law.

Does BatchPipes support connecting two Classic modules with an intervening
small Pipeline filter?  How?  Is a coordinated third job needed?

>The reasons are:  [Snip!  See the archive.]

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

The Business Case for BatchPipes in the z/OS Base (was: ... Pipes ...)

2021-09-27 Thread Hobart Spitz 
I'm going to pivot here.  I'm putting my support behind putting BatchPipes
in the z/OS base (rather than just Pipes).  If you agree, please
write/support such a requirement and/or educate your management to get
interested.  BatchPipes includes BatchPipesWorks, a not so current, but
still highly useful, version of TSO Pipelines.

The reasons are:

   1. The audience that could benefit from BatchPipes in the base is much
   larger than that of TSO Pipes in the base.  The former includes
   organizations that:
  - Have a limited batch window or want to reduce JOB run times.
  - Are constrained on developer resources.
  - Want to reduce hardware requirements.
  - Require quick resolution to production problems.
  - Have a policy to reduce their carbon footprint and/or their
  contribution to the climate crisis.
   2. It is unrealistic to expect wide buy-in for TSO Pipes since most z/OS
   sites are highly dependent on JCL.
   3. BatchPipes in the z/OS base would be the biggest enhancement to JCL
   since it (JCL) came into existence.
   4. BatchPipes in the base would improve the competitiveness of z/OS.
   5. IBM, vendors, and customers would benefit from the capability by
   being able to write quicker running, more general, less complicated JCL,
   knowing that BatchPipes would be available on all target systems.
   6. All parties could potentially benefit from having the PIPE command
   available on all target systems.
   7. BatchPipes fittings could enhance and extend existing programs and
   utilities with a consistent, intuitive, and uncomplicated control
   language.  Too many utilities have unique, inconsistent, and/or complex
   control languages.
   8. Contribute to global warming mitigation by reducing electricity usage
   due to processors, storage hardware and cooling.  This is more significant
   with BatchPipes as there is likely to be a larger impact in a shorter
   amount of time than with just Pipes alone.  The latter would be adopted
   more slowly and less broadly.  Climate crisis mitigation efforts may be
   exempt from the rumored requirement that IBM is legally barred from
   offering software at a loss.  (IMHO,  the huge number of competitors to IBM
   in today's market, suggests that any such requirement is obsolete and
   should be removed.)
   9. There would be some portability between z/VM and z/OS.
   10. Despite wide support, Pipes requirements have not budged for
   decades  .

Anyone so inclined, is welcome to submit such a requirement, adapt my Pipes
requirement, and/or work with me on a new requirement.  Of course, if you
agree, vote for the requirement.  Can the current Pipes requirement be
construed to support BatchPipes?  Is it too much to expect BatchPipes to be
added to the z/OS base without the delay of waiting for requirement(s),
voting, and acceptance.

Alternatively, BatchPipes and the REXX Compiler, possibly along with other
software, could be packaged together as a combined product, perhaps with a
performance or global warming mitigation theme .  Another possibility would
be to offer BatchPipes Light, where the intra-JOB piping was offered free
or in the base.  Half pipe fitting might be a loss-leader to attract
customers to the full product.

As a developer, I love REXX and I love Pipes.  Together, they are
programmer heaven, more so than OO.  Whenever I write something new,
whether in z/OS or z/VM, I write a REXX EXEC.  Under z/OS, I SPAWN the EXEC
when I want to run it in batch.

I have used BatchPipes only briefly, but I know Pipes fairly well.  Pipes
are the language of BatchPipes fittings, which essentially is an
enhancement that allows Pipes stages to run on most DDs between the program
and the storage media.

I don't like BatchPipes.  They require opsys or sysprog assistance and are
too much trouble to use.  They support inter-JOB piping which requires a
subsystem and makes the real valuable part (Half Pipe Fittings) complicated
and hard to use.  Since I almost never use JCL, I have no use for them.  I
can, however, see their use and, as listed above, their value to the z/OS
community as a whole.

That said, for the rest of this note, I will PRETEND to be someone with
production support responsibilities.  If some of these things have no
connection to anything that any site would actually do, I apologize.  I'm
just a color-blind painter trying to paint a picture that will give you the
idea.


In an imaginary future, some production control person may write:

I love BatchPipes!  Since we started using it, our batch production run
time has steadily dropped and is now just 20% of what it was.  We were able
to shorten the run-times of the longest running JOBs to a small fraction.
The DBAs love that they can start their maintenance and backups earlier and
with plenty of time to spare.

Much of the time, instead of writing a new