Re: COBOL and POSIX pipes (was: A design question)

[John McKown]

On Thu, Feb 16, 2017 at 7:49 PM, scott Ford  wrote:

> You guys are amazing. I havent used Unix on Z/OS much, a little. It has
> advantages for sure.
>
> Scott
>
>
​It adds some really nice features to legacy z/OS. But UNIX files can be
confusing to z/OS programmers because they are more like "memory" than
"disk" in that they are simply an ordered sequence of _bytes_, not
_records_. The file system itself does not have _any_ interpretation o​f
how those bytes are grouped into logical records. z/OS programmers, in
general, are used to reading individual records when reading a data set.
When you read a UNIX file, the program must tell the UNIX kernel ("access
method") how many bytes you want to read. UNIX will return __NO MORE__ than
that number of bytes. It could return fewer, if there are not that many
left before "end of file" (an on some other rare occasions). Your code must
then somehow know where the data you want (aka "this record") ends. Which
means either the file is composed of fixed length records, hard coded in
the program, or there is meta information encoded in the file data itself
which indicates a length (similar to the LLBB field in a z/OS variable
length data set) for each record. You can't rely on the system "handing"
you a "logical record: when you do a UNIX read simply because there is no
such thing, in a general sense.

The main drawback to z/OS UNIX is the relatively extreme overhead of doing
a fork() or spawn(). UNIX shells do those by the _ton_. Which explains the
BPXAS instances, which are the UNIX equivalent of a WLM batch initiator
(even runs the same program, IEFIIC, just with a special PARM= value).
Another drawback, to me, of z/OS UNIX is that the supplied UNIX tools (like
awk, find and /bin/sh) are _not_ as good as the GNU versions. Thankfully
Rocket Software has stepped into the gap with a nice port of many of them
for _no cost_, with optional paid support.

-- 
"Irrigation of the land with seawater desalinated by fusion power is
ancient. It's called 'rain'." -- Michael McClary, in alt.fusion

Maranatha! <><
John McKown

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: COBOL and POSIX pipes (was: A design question)

[scott Ford]

You guys are amazing. I havent used Unix on Z/OS much, a little. It has
advantages for sure.

Scott

On Mon, Feb 13, 2017 at 1:35 PM, John McKown 
wrote:

> On Mon, Feb 13, 2017 at 12:28 PM, Paul Gilmartin <
> 000433f07816-dmarc-requ...@listserv.ua.edu> wrote:
>
> > On Mon, 13 Feb 2017 11:58:57 -0600, John McKown wrote:
> > >
> > >...having already used
> > >a UNIX subroutine, I guess I figured it was easier to use another UNIX
> > >subroutine rather that setting up an FD along with an OPEN / READ loop /
> > >CLOSE.​
> > >
> > >... I think that the HLASM is basically
> > >being used as a "QSAM for RACF" type interface.​
> > >
> > If the replies returned by RACF are of uniform length or the assembler
> > program can pad them, BPX1RED is easiest.  I'm accustomed to
> > dealing with utility output where records are variable length,
> > contain no control characters, and are NL-terminated.  For that,
> > I might as well ALLOC and use the real QSAM.
> >
>
> ​Another point in your favor would be that the resultant code would be more
> understandable to a regular COBOL programmer. The only "oddity" would be
> the setup, in its calls to BPX1PIP and BPXWDYN.​
>
>
>
> >
> > -- gil
> >
> >
>
> --
> Our calculus classes are an integral part of your education.
>
> Maranatha! <><
> John McKown
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: COBOL and POSIX pipes (was: A design question)

[John McKown]

On Mon, Feb 13, 2017 at 12:28 PM, Paul Gilmartin <
000433f07816-dmarc-requ...@listserv.ua.edu> wrote:

> On Mon, 13 Feb 2017 11:58:57 -0600, John McKown wrote:
> >
> >...having already used
> >a UNIX subroutine, I guess I figured it was easier to use another UNIX
> >subroutine rather that setting up an FD along with an OPEN / READ loop /
> >CLOSE.​
> >
> >... I think that the HLASM is basically
> >being used as a "QSAM for RACF" type interface.​
> >
> If the replies returned by RACF are of uniform length or the assembler
> program can pad them, BPX1RED is easiest.  I'm accustomed to
> dealing with utility output where records are variable length,
> contain no control characters, and are NL-terminated.  For that,
> I might as well ALLOC and use the real QSAM.
>

​Another point in your favor would be that the resultant code would be more
understandable to a regular COBOL programmer. The only "oddity" would be
the setup, in its calls to BPX1PIP and BPXWDYN.​



>
> -- gil
>
>

-- 
Our calculus classes are an integral part of your education.

Maranatha! <><
John McKown

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: COBOL and POSIX pipes (was: A design question)

[Paul Gilmartin]

On Mon, 13 Feb 2017 11:58:57 -0600, John McKown wrote:
>
>...having already used
>a UNIX subroutine, I guess I figured it was easier to use another UNIX
>subroutine rather that setting up an FD along with an OPEN / READ loop /
>CLOSE.​
>
>... I think that the HLASM is basically
>being used as a "QSAM for RACF" type interface.​
> 
If the replies returned by RACF are of uniform length or the assembler
program can pad them, BPX1RED is easiest.  I'm accustomed to
dealing with utility output where records are variable length,
contain no control characters, and are NL-terminated.  For that,
I might as well ALLOC and use the real QSAM.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: COBOL and POSIX pipes (was: A design question)

[John McKown]

On Mon, Feb 13, 2017 at 11:51 AM, Paul Gilmartin <
000433f07816-dmarc-requ...@listserv.ua.edu> wrote:

> On Mon, 13 Feb 2017 11:32:36 -0600, John McKown wrote:
> >
> >
> ​
>
> >
> I was thinking of BPX1PIP and BPXWDYN( "alloc path('/dev/fd/[pipe
> output]') ..." ).
> No generated pathname.
>

​Ah. I hadn't thought of that. An interesting idea, but having already used
a UNIX subroutine, I guess I figured it was easier to use another UNIX
subroutine rather that setting up an FD along with an OPEN / READ loop /
CLOSE.​



>
> >> Does BPX1ATM dup() descriptors as spawn() does?  If so the parent must
> >> close() its copy of the input descriptor or it never sees EOF.  Happens
> to
> >> me all the time.
> >
> >​BPX1ATM is "attach_and_exec_mvs", which basically seems, to me, to be a
> >wrapper around the ATTACHX function. So the HLASM program is a subtask of
> >the COBOL program. ​
> >
> I take that as "doesn't dup()."
>

​Right, I should have said that explicitly. Why would it? What use would a
dup() serve?​ Both tasks (aka "threads") are running in the same UNIX
process and so (should?) have access to all of that process' file
descriptors.



>
> >​There is a BPX1SEL (
> >https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.
> 0/com.ibm.zos.v2r1.bpxb100/sel.htm?view=kc
> >) which I guess could be used. But, at least to me, using the BPX1ATM is
> >conceptually easier. ...
> >
> You need something like select() if you want to monitor multiple
> input streams, such as stdout and stderr from a child process.
>

​Yes, but that is not the case here. I think that the HLASM is basically
being used as a "QSAM for RACF" type interface.​


>
> -- gil
>
>
-- 
Our calculus classes are an integral part of your education.

Maranatha! <><
John McKown

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: COBOL and POSIX pipes (was: A design question)

[Paul Gilmartin]

On Mon, 13 Feb 2017 11:32:36 -0600, John McKown wrote:
>
>... The COBOL
>> >program would use BPX1RED to read from the read_file_descriptor. The HLASM
>> >
>> Is this better than using BPXWDYN('ALLOC ...') the descriptor and using
>> conventional COBOL I/O)?
>
>​Better? Not really. Just different. And doesn't require a MKFIFO with
>"random" name.​
> 
I was thinking of BPX1PIP and BPXWDYN( "alloc path('/dev/fd/[pipe output]') 
..." ).
No generated pathname.

>> Does BPX1ATM dup() descriptors as spawn() does?  If so the parent must
>> close() its copy of the input descriptor or it never sees EOF.  Happens to
>> me all the time.
>
>​BPX1ATM is "attach_and_exec_mvs", which basically seems, to me, to be a
>wrapper around the ATTACHX function. So the HLASM program is a subtask of
>the COBOL program. ​
> 
I take that as "doesn't dup()."

>​There is a BPX1SEL (
>https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.bpxb100/sel.htm?view=kc
>) which I guess could be used. But, at least to me, using the BPX1ATM is
>conceptually easier. ...
> 
You need something like select() if you want to monitor multiple
input streams, such as stdout and stderr from a child process.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: COBOL and POSIX pipes (was: A design question)

[John McKown]

On Mon, Feb 13, 2017 at 10:58 AM, Paul Gilmartin <
000433f07816-dmarc-requ...@listserv.ua.edu> wrote:

> On Mon, 13 Feb 2017 09:44:51 -0600, John McKown wrote:
> >
> >> 3.  Is it possible to have the Assembler routine place the extracted
> data into a unix
> >>  pipe or a C type=memory file and then i can retrieve it ?
> >
> >​You can call the UNIX BPX* routines from COBOL, so you can use something
> >like BPX1PIP to create an unnamed pipe. The COBOL program could then ​pass
> >the "write_file_descriptor" to the HLASM program as a parm. The COBOL
> >program would use BPX1RED to read from the read_file_descriptor. The HLASM
> >
> Is this better than using BPXWDYN('ALLOC ...') the descriptor and using
> conventional COBOL I/O)?
>

​Better? Not really. Just different. And doesn't require a MKFIFO with
"random" name.​



>
> >would use BPX1WRT to write to the passed in descriptor. But critically
> >important is that the HLASM program must be ATTACHX'd (BPX1ATM service can
> >do this) and not LINK'd to (as a normal COBOL CALL would do). Otherwise
> >
> Does BPX1ATM dup() descriptors as spawn() does?  If so the parent must
> close() its copy of the input descriptor or it never sees EOF.  Happens to
> me all the time.
>

​BPX1ATM is "attach_and_exec_mvs", which basically seems, to me, to be a
wrapper around the ATTACHX function. So the HLASM program is a subtask of
the COBOL program. ​



>
> >you'll most likely get a lock out with either the TCB either waiting in
> the
> >COBOL code for the HLASM to write some thing, or in the HLASM code waiting
> >(buffer full) for the COBOL to read something. I just wanted to throw this
> >last in for completeness. You might also need some way to communicate to
> >the COBOL routine that the HLASM routine has written an "EOF" and has
> ended.
> >
> close() the input end of the pipe should suffice.  COBOL should see EOF.
>

​Ah. I hadn't thought of that. Thanks for the info.


>
>
> On Mon, 13 Feb 2017 16:45:33 +0100, Peter Hunkeler wrote:
> >
> >Any you'd need to run the Cobol and the assembler routine in parallel,
> not as main routine calling a subroutine. UNIX pipes need both ends to be
> acitve for them to work.
> >
> I've sometimes cheated on this, mostly with stderr, assuming that
> any error messages will fit in a 131KB buffer.  Hard to handle both
> stdout and stderr otherwise in a Rexx parent.  Is there a select()
> SYSCALL?
>

​There is a BPX1SEL (
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.bpxb100/sel.htm?view=kc
) which I guess could be used. But, at least to me, using the BPX1ATM is
conceptually easier. Mainly because the HLASM routine would not need to
"return" to the COBOL routine except at the very end. Otherwise, Scott
could just do a normal CALL to "get the next record" and have the HLASM
routine be able to "read next" the RACF information, which would mean
saving "positioning" information over the CALL. I also got the impression
that the HLASM and COBOL are pretty much already written at using a
"socket" type interface is mainly to eliminate the use of an intermediate
"disk file buffer". It would also enhance security some because the data
would never make it to a disk device, just memory.

I was also thinking of the COBOL program passing a record buffer address to
the HLASM program, then coordinating access to the buffer using a semaphone
(may require two?) (on == ready to read; off == ready to write), but that
looked a bit more difficult to me.


>
> -- gil
>
>


-- 
Our calculus classes are an integral part of your education.

Maranatha! <><
John McKown

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

COBOL and POSIX pipes (was: A design question)

[Paul Gilmartin]

On Mon, 13 Feb 2017 09:44:51 -0600, John McKown wrote:
>
>> 3.  Is it possible to have the Assembler routine place the extracted data 
>> into a unix
>>  pipe or a C type=memory file and then i can retrieve it ?
>
>​You can call the UNIX BPX* routines from COBOL, so you can use something
>like BPX1PIP to create an unnamed pipe. The COBOL program could then ​pass
>the "write_file_descriptor" to the HLASM program as a parm. The COBOL
>program would use BPX1RED to read from the read_file_descriptor. The HLASM
>
Is this better than using BPXWDYN('ALLOC ...') the descriptor and using
conventional COBOL I/O)?

>would use BPX1WRT to write to the passed in descriptor. But critically
>important is that the HLASM program must be ATTACHX'd (BPX1ATM service can
>do this) and not LINK'd to (as a normal COBOL CALL would do). Otherwise
>
Does BPX1ATM dup() descriptors as spawn() does?  If so the parent must
close() its copy of the input descriptor or it never sees EOF.  Happens to
me all the time.

>you'll most likely get a lock out with either the TCB either waiting in the
>COBOL code for the HLASM to write some thing, or in the HLASM code waiting
>(buffer full) for the COBOL to read something. I just wanted to throw this
>last in for completeness. You might also need some way to communicate to
>the COBOL routine that the HLASM routine has written an "EOF" and has ended.
> 
close() the input end of the pipe should suffice.  COBOL should see EOF.


On Mon, 13 Feb 2017 16:45:33 +0100, Peter Hunkeler wrote:
>
>Any you'd need to run the Cobol and the assembler routine in parallel, not as 
>main routine calling a subroutine. UNIX pipes need both ends to be acitve for 
>them to work.
> 
I've sometimes cheated on this, mostly with stderr, assuming that
any error messages will fit in a 131KB buffer.  Hard to handle both
stdout and stderr otherwise in a Rexx parent.  Is there a select()
SYSCALL?

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN