Re: CMSFS package for Linux release 1.1.12

[Philip Tully]

phlip.tu...@citi.com
" virtus in medio stat "
"Perfect is the enemy of the Good"


On Sun, Feb 4, 2024 at 7:55 PM Rick Troth  wrote:

> Hi all --
>
> The first CMS FS package (a long long time ago, but I can still
> remember) ... okay, mute Don McLean.
> The first CMS FS package included a filesystem driver. That fell out
> because the author could not keep up with Linux kernel changes. (It's a
> rapid pace!) And now there's the FUSE driver.
> But the package also includes a utility which lets you read CMS
> minidisks (and read files from them) without having to mount them.
> Sometimes quite handy.
>
> I've occasionally used a feature called 'cmsfsrun'. This little widget
> lets you run a shell script from your 191 when Linux starts up.
> But it was never published with the rest of the project. As of 1.1.12
> that has been corrected.
> The default CMS file is "profile sh" but that is configurable. Give it a
> try and let me know what you think.
> It works with the SysV INIT stuff but there is also a "service" file for
> SystemD.
>
> https://github.com/trothr/cmsfs/releases/tag/1.1.12
>
> There is also an RPM, which is helpful for SUSE and RedHat systems. (If
> someone wants to help with.deb files please speak up.)
>
> This is a GitHub thing. Feel free to open "issues" if you find any. Thanks.
>
> -- R; <><
>
>
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www2.marist.edu/htbin/wlvindex?LINUX-390
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

CMSFS package for Linux release 1.1.12

[Rick Troth]

Hi all --

The first CMS FS package (a long long time ago, but I can still
remember) ... okay, mute Don McLean.
The first CMS FS package included a filesystem driver. That fell out
because the author could not keep up with Linux kernel changes. (It's a
rapid pace!) And now there's the FUSE driver.
But the package also includes a utility which lets you read CMS
minidisks (and read files from them) without having to mount them.
Sometimes quite handy.

I've occasionally used a feature called 'cmsfsrun'. This little widget
lets you run a shell script from your 191 when Linux starts up.
But it was never published with the rest of the project. As of 1.1.12
that has been corrected.
The default CMS file is "profile sh" but that is configurable. Give it a
try and let me know what you think.
It works with the SysV INIT stuff but there is also a "service" file for
SystemD.

https://github.com/trothr/cmsfs/releases/tag/1.1.12

There is also an RPM, which is helpful for SUSE and RedHat systems. (If
someone wants to help with.deb files please speak up.)

This is a GitHub thing. Feel free to open "issues" if you find any. Thanks.

-- R; <><



--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Permissions of mounted CMS disks after cmsfs-fuse

[MacIsaac, Michael (CORP)]

Hello list,

I need to get CMS files to Linux as a non-root user (zadmin in this case).  I 
am able to mount the CMS disk as such (had to add the -o umask flag):
  # sudo cmsfs-fuse -a -o umask=022 /dev/dasdi /mnt/CMSdiskRO

If I look at the permissions as root, they look fine - the directory is world 
readable and executable:
  # cd /mnt
  root@cdlenghil101:/mnt # la
  total 12
  drwxr-xr-x  19 root root   267 Dec  2 09:33 ../
  drwxr-x---   2 root root 6 Dec  3 07:13 tmp/
  drwxr-xr-x.  4 root root34 Dec  3 07:13 ./
  drwxr-xr-x   2 root root 10176 Dec  8 06:46 CMSdiskRO/

But if I try to look at the same directory as zadmin, I get this:
  # cd /mnt
  # ls -la
  ls: cannot access 'CMSdiskRO': Permission denied
  total 0
  drwxr-xr-x.  4 root root  34 Dec  3 07:13 ./
  drwxr-xr-x  19 root root 267 Dec  2 09:33 ../
  d??  ? ??  ?? CMSdiskRO/
  drwxr-x---   2 root root   6 Dec  3 07:13 tmp/

And zadmin is unable to copy any of the mounted files.

Could this be a bug in cmsfs-fuse?

Any help will be appreciated.

Thanks.

-Mike M




--
This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, notify the sender immediately by return email and delete the message 
and any attachments from your system.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: Permissions of mounted CMS disks after cmsfs-fuse

[Michael MacIsaac]

Thanks bear, that worked.

I could argue that ignoring the umask is a bug, but I'd probably get
"working as designed", so I won't bother to open a case.

-Mike M

On Fri, Jan 8, 2021 at 3:32 PM r.stricklin  wrote:

> On Jan 8, 2021, at 8:52 AM, MacIsaac, Michael (CORP) wrote:
>
> > Hello list,
> >
> > I need to get CMS files to Linux as a non-root user (zadmin in this
> case).  I am able to mount the CMS disk as such (had to add the -o umask
> flag):
> >  # sudo cmsfs-fuse -a -o umask=022 /dev/dasdi /mnt/CMSdiskRO
>
> fuse is user-space. It wouldn't surprise me to learn that a fuse-mounted
> filesystem is only usable by the user who mounted it.
>
> indeed, this would seem the reason for `-o allow_other`
>
> ok
> bear.
>
>
> --
> until further notice
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www2.marist.edu/htbin/wlvindex?LINUX-390
>


--
 -Mike MacIsaac

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

Re: Permissions of mounted CMS disks after cmsfs-fuse

[r.stricklin]

On Jan 8, 2021, at 8:52 AM, MacIsaac, Michael (CORP) wrote:

> Hello list,
> 
> I need to get CMS files to Linux as a non-root user (zadmin in this case).  I 
> am able to mount the CMS disk as such (had to add the -o umask flag):
>  # sudo cmsfs-fuse -a -o umask=022 /dev/dasdi /mnt/CMSdiskRO

fuse is user-space. It wouldn't surprise me to learn that a fuse-mounted 
filesystem is only usable by the user who mounted it.

indeed, this would seem the reason for `-o allow_other`

ok
bear.


-- 
until further notice

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www2.marist.edu/htbin/wlvindex?LINUX-390

CMSFS utility

[Rick Troth]

Good questions, Paul.
Copying the Linux-390 discussion list because it seems relevant.

I'll discuss the filesystem driver ("cmsfs.o" below) in a separate note,
but the CMSFS utility was written for maximum portability. Got an update
from Dan Horák a year or five ago, and that should be in the github
source. Otherwise, it has been stable. Credit the fine folks at Endicott
for making rock-solid sub-systems.


On 08/25/2017 07:58 AM, Paul Flint wrote:
>> So ... all I/O on z is done in this uniform hexadecimal space. Very
>> nice. (And forgive me if I'm telling you things you already know.)
>
> Yea but you make it sound so easy...

SMOP, and systems like Linux have the device drivers, so it /is/ easy.


>
>> Disks ... virtual machine gotta have disks just like a real machine.
>> CMS has some standard addresses: 190 is the system disk, 191 is
>> your personal disk. CMS also uses 181 for your personal tape drive.
>> There's a progression: 18x, 19x, 1Ax, 1Bx. The 1A0 and 1A1 disks
>> might be used for your own VM-on-VM (that's a whole nutha story),
>> so I put NORD at 1B0 and 1B1.
>
> This is good stuff.

There used to be more.
CMS has 19E and "we" used to put Linux /usr at 1BE.
But the KISS principle trumps trickery and simpler is sweeter.


>
>> 1B1 is nothing more or less than your root disk when you IPL NORD.
>> As configured at Dave's place, NORD uses a shared segment for the op
>> sys, so you don't strictly need the 1B0. But it's there and gets used
>> when doing maint.
>
> Since this is all about volumes I will mention my bumbling in this
> area here and my bumbling on the network front in another note.

It's all good.
(And for the rest, know that Paul is online. Just a little NAT action by
The Generous and FLINT flies fine.)


> The good news:
> #
> CP Q V DASD
> DASD 0190 3390 M01RES R/O214 CYL ON DASD  2002 SUBCHANNEL = 0004
> DASD 0191 3390 USR006 R/W500 CYL ON DASD  1006 SUBCHANNEL = 000B
> DASD 019D 3390 M01RES R/O292 CYL ON DASD  2002 SUBCHANNEL = 0005
> DASD 019E 3390 M01RES R/O500 CYL ON DASD  2002 SUBCHANNEL = 0006
> DASD 019F 3390 USR001 R/O150 CYL ON DASD  1000 SUBCHANNEL = 0007
> DASD 01B0 3390 USR00A R/W   2000 CYL ON DASD  1009 SUBCHANNEL = 0009
> DASD 01B1 3390 USR00B R/W   2000 CYL ON DASD  100A SUBCHANNEL = 000A

Standard set. Looks good.


> From your 2000 github:
>
> 'cmsfsvol'  report on a CMS volume in CMS 'Q DISK' format
> 'cmsfslst'  list CMS files in CMS 'LISTFILE' format
> 'cmsfscat'  conCATenate one or more CMS files to stdout
> 'cmsfsck' - filesystem check for CMS vols (aka: fsck.cmsfs)
> 'cmsfscp' - copy CMS files, preserving time stamps
> libcmsfs.a  "archive" library for user space (utility) mode
> cmsfs.o --- the FS driver, for 'mount -t cms' to work
>
> Shear guesses as to how to use the first in the list:
>
>  cmsfsvol
>  /usr/bin/cmsfslst /dev/dasdb1
>  /usr/bin/cmsfsvol dev/dasdb1
>  /usr/bin/cmsfsvol 1b1
>  /usr/bin/cmsfsvol 01b1
>  /usr/bin/cmsfsvol 01B1
>
> Maybe?

Ah! To partition or not to partition. That is the question.
In Linux parlance, the trailing "1" indicates "partition one". But CMS
minidisks are not partitioned, so the FS goes on "the whole disk".

But also, the CMSFS commands are useless against your 1Bx disks. You
need CMS format minidisks, the 19x range. Also, your 191 will not beyour
present /dev/dasda or /dev/dasdb. And  they're not explicitly
online. Let's fix that.

Finally, you're on NORD which doesn't have UDEV (or at least doesn't
have UDEV yet). So there are manual steps for bringing the 191 online.
(This is /not/ rocket surgery.) Bear with me ...

echo 1 > /sys/bus/ccw/devices/0.0.0191/online
# to bring your 191 online

/etc/nordblks.sh
# to fudge for lack of UDEV and create the device file(s)

ls -lat /sys/bus/ccw/devices/0.0.0191/block
# to know where the kernel slotted your 191

cmsfsvol /dev/dasdq
# to see the CMS  'q disk a'  equivalent


And there you have it!
The above steps are scripted in /etc/init.d/cmsfsrun on your guest, but
it has its own purpose, not general.

If you had 190-19F in your boot parms, they'd be online from the get-go,
and you could rely on /dev/dasdq (or whatever, probably /dev/dasdr in
that case) always bein there.


> # /usr/bin/cmsfsvol USR00A
> LABEL  VDEV M  STAT   CYL TYPE BLKSZ   FILES  BLKS USED-(%) BLKS LEFT 
> BLK TOTAL
>
> open(): No such file or directory
> cannot open 'USR00A'
> 'USR00A' is not a CMS volume.
>
> Could use a hint...

If "USR00A" was a plain file (with a CMS FS disk image) or a device file
(of a CMS minidisk) or a sym-link to either, t

Re: [EXTERNAL] Re: cmsfs-fuse data refreshing

[Isaac Raschkovsky]

Thanks Alan


-Original Message-
From: Linux on 390 Port [mailto:LINUX-390@VM.MARIST.EDU] On Behalf Of Alan 
Altmark
Sent: Wednesday, August 17, 2016 9:58 PM
To: LINUX-390@VM.MARIST.EDU
Subject: [EXTERNAL] Re: cmsfs-fuse data refreshing

No, because there is no logical connection between the Linux guest and the CMS 
guest.  Linux has no way to know that the disk contents were changed.

Only by using NFS can Linux see a changed file, and that file would be an SFS 
file.

Regards,
  Alan


   Isaac Raschkovsky --- [LINUX-390] cmsfs-fuse data refreshing ---
From:"Isaac Raschkovsky" 
<isaac.raschkov...@gm.com>To:LINUX-390@VM.MARIST.EDUDate:Wed, Aug 17, 2016 2:44 
PMSubject:[LINUX-390] cmsfs-fuse data refreshing

Is there any way that we can make changes or create a file on CMS and 
access the changes on a cmsfs-fuse directory already mounted?  It looks like 
the data is not refreshed on Linux and I  have to unmount vary the dasd 
offline, vary online and mount again to see the changes.Any suggestion would be 
appreciate. ThanksIsaacNothing in this message is intended to constitute an 
electronic signature unless a specific statement to the contrary is included in 
this message.Confidentiality Note: This message is intended only for the person 
or entity to which it is addressed. It may contain confidential and/or 
privileged material. Any review, transmission, dissemination or other use, or 
taking of any action in reliance upon this message by persons or entities other 
than the intended recipient is prohibited and may be unlawful. If you received 
this message in error, please contact the sender and delete it from your 
computer.--For
 LINUX-390 subscribe / signoff / archive access instructions,send email to 
lists...@vm.marist.edu with the message: INFO LINUX-390 or 
visithttp://www.marist.edu/htbin/wlvindex?LINUX-390--For
 more information on Linux on System z, visithttp://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions, send email to 
lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit http://wiki.linuxvm.org/


Nothing in this message is intended to constitute an electronic signature 
unless a specific statement to the contrary is included in this message.

Confidentiality Note: This message is intended only for the person or entity to 
which it is addressed. It may contain confidential and/or privileged material. 
Any review, transmission, dissemination or other use, or taking of any action 
in reliance upon this message by persons or entities other than the intended 
recipient is prohibited and may be unlawful. If you received this message in 
error, please contact the sender and delete it from your computer.

Re: cmsfs-fuse data refreshing

[Alan Altmark]

No, because there is no logical connection between the Linux guest and the CMS 
guest.  Linux has no way to know that the disk contents were changed.

Only by using NFS can Linux see a changed file, and that file would be an SFS 
file.

Regards,
  Alan


   Isaac Raschkovsky --- [LINUX-390] cmsfs-fuse data refreshing --- 
From:"Isaac Raschkovsky" 
<isaac.raschkov...@gm.com>To:LINUX-390@VM.MARIST.EDUDate:Wed, Aug 17, 2016 2:44 
PMSubject:[LINUX-390] cmsfs-fuse data refreshing
  
Is there any way that we can make changes or create a file on CMS and 
access the changes on a cmsfs-fuse directory already mounted?  It looks like 
the data is not refreshed on Linux and I  have to unmount vary the dasd 
offline, vary online and mount again to see the changes.Any suggestion would be 
appreciate. ThanksIsaacNothing in this message is intended to constitute an 
electronic signature unless a specific statement to the contrary is included in 
this message.Confidentiality Note: This message is intended only for the person 
or entity to which it is addressed. It may contain confidential and/or 
privileged material. Any review, transmission, dissemination or other use, or 
taking of any action in reliance upon this message by persons or entities other 
than the intended recipient is prohibited and may be unlawful. If you received 
this message in error, please contact the sender and delete it from your 
computer.--For
 LINUX-390 subscribe / signoff / archive access instructions,send email to 
lists...@vm.marist.edu with the message: INFO LINUX-390 or 
visithttp://www.marist.edu/htbin/wlvindex?LINUX-390--For
 more information on Linux on System z, visithttp://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

cmsfs-fuse data refreshing

[Isaac Raschkovsky]

Is there any way that we can make changes or create a file on CMS and access 
the changes on a cmsfs-fuse directory already mounted?  It looks like the data 
is not refreshed on Linux and I  have to unmount vary the dasd offline, vary 
online and mount again to see the changes.

Any suggestion would be appreciate. Thanks

Isaac


Nothing in this message is intended to constitute an electronic signature 
unless a specific statement to the contrary is included in this message.

Confidentiality Note: This message is intended only for the person or entity to 
which it is addressed. It may contain confidential and/or privileged material. 
Any review, transmission, dissemination or other use, or taking of any action 
in reliance upon this message by persons or entities other than the intended 
recipient is prohibited and may be unlawful. If you received this message in 
error, please contact the sender and delete it from your computer.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[WF Konynenberg]

On 05/24/2016 03:40 AM, Alan Altmark wrote:

On Monday, 05/23/2016 at 08:30 GMT, WF Konynenberg <w...@konynenberg.org>
wrote:

My concern was that Alan Altmark suggested that a critical bit of
information to positively identify these files was not available to
Linux programs, which is apparently not the case.

I meant that the attributes (user.record_format and user.record_lrecl)
aren't expressed by the driver as a "normal" file attribute that a
"normal" Linux application might be expected to extract on any file.   I
assumed (but did not verify!) that if you copied the file from the CMS
disk to a native Linux file, the extended attributes would be lost.  (I
don't know if extended attributes are a run-time conjuring of the file
system driver or if, once created, they persist with the file forever,
even if copied to a non-cmsfs-fuse file system.)

If they do persist, then a cmspack utility would work as well as CMS
COPYFILE.



Whether such extended attributes are copied along with the file depends
on a) whether the target file system supports extended attributes (most
Linux file systems do), and b) whether the user asked for them to be
copied.  By default, the common cp command will not copy extended
attributes.  You have to ask for that to happen. The simplest way is
perhaps to do "cp -a ..." to invoke archive mode.  Other copying tools
may or may not be a bit more careful to by default copy relevant
metadata along.


Willy

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Alan Altmark]

On Monday, 05/23/2016 at 08:30 GMT, WF Konynenberg <w...@konynenberg.org> 
wrote:
> My concern was that Alan Altmark suggested that a critical bit of
> information to positively identify these files was not available to
> Linux programs, which is apparently not the case.

I meant that the attributes (user.record_format and user.record_lrecl) 
aren't expressed by the driver as a "normal" file attribute that a 
"normal" Linux application might be expected to extract on any file.   I 
assumed (but did not verify!) that if you copied the file from the CMS 
disk to a native Linux file, the extended attributes would be lost.  (I 
don't know if extended attributes are a run-time conjuring of the file 
system driver or if, once created, they persist with the file forever, 
even if copied to a non-cmsfs-fuse file system.)

If they do persist, then a cmspack utility would work as well as CMS 
COPYFILE.

Alan Altmark

Senior Managing z/VM and Linux Consultant
Lab Services System z Delivery Practice
IBM Systems & Technology Group
ibm.com/systems/services/labservices
office: 607.429.3323
mobile; 607.321.7556
alan_altm...@us.ibm.com
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Rick Troth]

On 05/21/2016 09:58 AM, WF Konynenberg wrote:
> Aye, from what I have read about these packed files here, it seems to
> me to be much like the various compressed file formats in use on
> Linux/Unix (.z, .gz, .bz, .xz, etc), except that perhaps the CMS pack
> tool neglects to modify the file name extension to indicate that this
> is now a packed file.  That means the "native" way to deal with these
> on Linux would be to implement some "cmspack" utility which can
> pack/unpack such files (as was indicated, the assembler source code is
> available so it's easy to figure out how to do this and reimplement it
> in a bit of C code), and teach the file command about this file format
> so it can correctly report it.

Filename extensions are not available to CMS (actual name of the
filesystem is EDF). But you were talking about something else, I
understand.


On 05/22/2016 01:12 AM, Alan Altmark wrote:
> There is no 'neglect'. Packed-ness is not tied to the filetype.

And that's a Good Thing. [tm]

Do one thing and do it well.


On 05/22/2016 07:22 PM, Mark Post wrote:
> Again, Alan's original suggestion was to teach cmsfs-fuse about such things.

Which I think is where Willy's heading. Sort of. (see below)


> I suppose one could use a combination of cmsfs-fuse and Rick Troth's 
> wonderful CMSFS package.

Thank you for the kind words, sir! I owe you yet another beer at
SCIDS.   :-)
Or maybe lunch at Chuy's in Austin ... something more substantial.

By the way, CMS FS (the older one) is now a Github project
<https://github.com/trothr/cmsfs> if anyone is interested. Contributions
welcome!
The FUSE based driver has surpassed it (can do read-write and actually
still works) but the older one has a utility mode which runs on any Unix.



On 05/22/2016 09:01 PM, WF Konynenberg wrote:
>   ...
> I'm just saying that any information that's available in the CMS file
> system "out of band" that is somehow relevant to an application trying
> to read/write any given file must be made available to the Linux
> programs in some way or other.   ...

Sure ... and RECFM and LRECL are available to CMS FS (either flavor) so
within a filesystem driver (or utility) a combination of RECFM=F and
LRECL=1024 with the magic numbers Alan posted would be strong indication
"this is in CMS Packed format".


>
> Cmsfs-fuse could provide this out-of-band info in various ways.

It could present a different Linux side filename. (with appropriate
extension tacked on)


> There's
> no "perfect" solution.   ...

Too true.

But something else:
Since all FUSE is user-land anyway, there's no reason why it should not
call out some user-land helper.
Maybe cms-fuse could call an unpacker, verbs like Hendrik mentioned.
(And then leave the fn.ft as-is.)

This violates "do one thing and do it well" (Mark alluded to,
consciously or not, saying filesystems typically don't take on these
tasks). But again, -FUSE is user-land. (We're not in Kernel anymore,
Toto.)
Just thinkin.


-- R; <><




--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[David Kreuter]

Mark - 
It is the fm letter in use when the CMS RELEASE command was last done
from the writing virtual machine for new or changed files. On disk the
fm letter is meaningless while the fm number does have meaning and needs
preservation. 
David


 Original Message 
Subject: Re: Reading VMLOGS files from Linux (cmsfs-fuse)
From: Mark Post <mp...@suse.com>
Date: Mon, May 23, 2016 1:28 pm
To: LINUX-390@VM.MARIST.EDU

>>> On 5/22/2016 at 07:22 PM, Mark Post wrote: 
> s390vsl204:~ # cmsfslst -f /dev/dasdg
> FILENAME FILETYPE FM FORMAT LRECL RECS BLOCKS DATE TIME
> DIRECTOR P0 F 64 8 1 5/04/2016 19:56:38
> ALLOCMAP P0 F 4096 2 2 5/04/2016 19:56:38
> FTPBOOT PARM B1 F 80 2 1 5/04/2016 18:07:40
> S11S2GM EXEC Z1 F 80 25 1 10/01/2012 6:23:36
> KAKKA2 EXEC Z1 F 80 25 1 10/01/2012 6:27:42
> KAKKA3 EXEC Z1 F 80 25 1 10/01/2012 6:28:18
> LINUX EXEC Z1 F 80 25 1 10/01/2012 6:27:00
> INITRAMF IMG B1 F 80 435159 8500 5/04/2016 19:56:38

I just noticed something unexpected in this output I posted yesterday.
How can files on one minidisk have different filemode letters? B1 _and_
Z1? For that matter, how can there be a letter associated at all, since
it's not being ACCESSed by CMS? I would suspect that cmsfslst is
assigning arbitrary values to things, but getfattr shows the same
values.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Mark Post]

>>> On 5/22/2016 at 07:22 PM, Mark Post wrote: 
> s390vsl204:~ # cmsfslst -f /dev/dasdg
> FILENAME FILETYPE FM FORMAT LRECL   RECS BLOCKS DATE TIME
>  DIRECTOR P0 F 64  8  1  5/04/2016 19:56:38
>  ALLOCMAP P0 F   4096  2  2  5/04/2016 19:56:38
> FTPBOOT  PARM B1 F 80  2  1  5/04/2016 18:07:40
> S11S2GM  EXEC Z1 F 80 25  1 10/01/2012  6:23:36
> KAKKA2   EXEC Z1 F 80 25  1 10/01/2012  6:27:42
> KAKKA3   EXEC Z1 F 80 25  1 10/01/2012  6:28:18
> LINUXEXEC Z1 F 80 25  1 10/01/2012  6:27:00
> INITRAMF IMG  B1 F 80 435159   8500  5/04/2016 19:56:38

I just noticed something unexpected in this output I posted yesterday.  How can 
files on one minidisk have different filemode letters?  B1 _and_ Z1?  For that 
matter, how can there be a letter associated at all, since it's not being 
ACCESSed by CMS?  I would suspect that cmsfslst is assigning arbitrary values 
to things, but getfattr shows the same values.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[WF Konynenberg]

On 05/23/2016 09:46 AM, Hendrik Brueckner wrote:

On Sun, May 22, 2016 at 05:22:56PM -0600, Mark Post wrote:

On 5/22/2016 at 10:45 AM, WF Konynenberg <w...@konynenberg.org> wrote:



The cmsfs-fuse also provides information the file mode and format.  They can
be obtained with getfattr or even set with setfattr.  This kind of
meta-information is managed as extended file attributes.

If there is some indication of packed CMS in these kind of meta-information it
could help to spot them in Linux.  Of course, if cmsfs-fuse is missing some
meta-information it should display with getfattr, please let us know.

 From a Linux perspective, I would still consider packed CMS files like any
other compressed files like bz2, xz, ...  So having a magic to stop these and,
optionally, a cmspack/cmsunpack command would help.


Aha,

So when Alan Altmark wrote that LRECL and RECFM aren't available to
applications, he didn't consider these attributes.  This renders the
entire discussion moot, as the relevant meta data is apparently already
available to Linux programs through a suitable mechanism.  (you only
mention mode & format, but I gather that includes LRECL...)

And yes, based on what I have heard so far about how these files appear
and are accessed in CMS, they would seem comparable to .gz, .xz etc
files and thus it would make sense to simply provide a similar
(de)compression tool for them and add their magic numbers to the file
program.

My concern was that Alan Altmark suggested that a critical bit of
information to positively identify these files was not available to
Linux programs, which is apparently not the case.

Thanks for clearing this up.

Willy

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Hendrik Brueckner]

On Sun, May 22, 2016 at 05:22:56PM -0600, Mark Post wrote:
> >>> On 5/22/2016 at 10:45 AM, WF Konynenberg <w...@konynenberg.org> wrote:
> > In Linux/Unix, a file is simply a byte stream.  Any format information
> > about a structured file should be accessible to the application by some
> > means, so it can know how to parse the file. Since the CMS file system
> > stores structured files of different types/modes, this meta information
> > in the file system should somehow be made available to applications in
> > Linux/Unix when accessing the file system through cms-fuse.  There are
>
> Sounds like you're starting to argue Alan's case here.  Have the file system
> driver do the heavy lifting for you.  I don't think that's likely to happen.
>
> > various ways to achieve that, e.g. represent each CMS file by 2 files in
> > Linux/Unix (similar, I think, to how "resource forks" are handled in
> > Apple systems), one containing the data, one containing the meta data,
> > or, alternatively, add a header to each file that represents the meta
> > data from the file system (though this might break some existing
> > applications that currently expect to find only the actual data in the
> > file and somehow "magically" know the meta data...), etc, etc. Finding
> > the most suitable model will likely require some creative thought on the
> > part of the developers.  Perhaps the representation model can be made to
> > depend on a mount option, with the default being backward compatible
> > lack of metadata.
> >
> > Just simply saying "sorry, this critical meta data from the CMS file
> > system is not available in Linux" basically means unnecessarily
> > crippling Linux access to the CMS file system.
>
> Again, Alan's original suggestion was to teach cmsfs-fuse about such things.
>
> I suppose one could use a combination of cmsfs-fuse and Rick Troth's 
> wonderful CMSFS package.
>
> s390vsl204:~ # cmsfs-fuse /dev/dasdg /mnt
> s390vsl204:~ # l /mnt
> total 34024
> drwxrwx--- 2 root root  512 May  4 18:56 ./
> drwxr-xr-x 1 root root  172 May 18 19:00 ../
> -rw-rw 1 root root  160 May  4 17:07 FTPBOOT.PARM
> -rw-rw 1 root root 34812720 May  4 18:56 INITRAMF.IMG
> -rw-rw 1 root root 2000 Oct  1  2012 KAKKA2.EXEC
> -rw-rw 1 root root 2000 Oct  1  2012 KAKKA3.EXEC
> -rw-rw 1 root root 2000 Oct  1  2012 LINUX.EXEC
> -rw-rw 1 root root 2000 Oct  1  2012 S11S2GM.EXEC
>
> s390vsl204:~ # cmsfslst -f /dev/dasdg
> FILENAME FILETYPE FM FORMAT LRECL   RECS BLOCKS DATE TIME
>  DIRECTOR P0 F 64  8  1  5/04/2016 19:56:38
>  ALLOCMAP P0 F   4096  2  2  5/04/2016 19:56:38
> FTPBOOT  PARM B1 F 80  2  1  5/04/2016 18:07:40
> S11S2GM  EXEC Z1 F 80 25  1 10/01/2012  6:23:36
> KAKKA2   EXEC Z1 F 80 25  1 10/01/2012  6:27:42
> KAKKA3   EXEC Z1 F 80 25  1 10/01/2012  6:28:18
> LINUXEXEC Z1 F 80 25  1 10/01/2012  6:27:00
> INITRAMF IMG  B1 F 80 435159   8500  5/04/2016 19:56:38
>
> I don't think the file command is going to be extended to handle this, 
> however.

The cmsfs-fuse also provides information the file mode and format.  They can
be obtained with getfattr or even set with setfattr.  This kind of
meta-information is managed as extended file attributes.

If there is some indication of packed CMS in these kind of meta-information it
could help to spot them in Linux.  Of course, if cmsfs-fuse is missing some
meta-information it should display with getfattr, please let us know.

>From a Linux perspective, I would still consider packed CMS files like any
other compressed files like bz2, xz, ...  So having a magic to stop these and,
optionally, a cmspack/cmsunpack command would help.

Thanks and kind regards,
  Hendrik

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Alan Altmark]

On Monday, 05/23/2016 at 02:41 GMT, Mark Post  wrote:
> I recently came across a question in an online forum where the poster 
was asking how to tell if the
> input their program was reading was base64 encoded.  The various 
responders were suggesting all
> sorts of things they could check to get some idea if it might be or not. 
 My suggestion would have
> been "try to decode it and see if it works.  If it does, it was base64 
encoded."  I suspect
> something similar is going to wind up having to be done here.

XEDIT reads the entire file into memory.  If at any time during that 
process, there's a glitch in unpacking the file, XEDIT abandons the 
unpacking and assumes the file wasn't packed after all.

Alan Altmark

Senior Managing z/VM and Linux Consultant
Lab Services System z Delivery Practice
IBM Systems & Technology Group
ibm.com/systems/services/labservices
office: 607.429.3323
mobile; 607.321.7556
alan_altm...@us.ibm.com
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Mark Post]

>>> On 5/22/2016 at 09:01 PM, WF Konynenberg <w...@konynenberg.org> wrote: 
> So if cmsfslst can find all the relevant bits & bytes in the CMS file
> system that indicate what sort of file this is, and how it is to be
> read/parsed by programs, then cmsfs-fuse should be able to somehow (by
> whatever feasible means) pass the info from these bits & bytes to the
> Linux program accessing the file system through cmfsf-fuse.

Which is typically not done by the various file systems because the usual 
intent is that no real changes will have to be made to the various tools that 
people are accustomed to using.  Things like ls, cat, file, vi, etc., etc.  The 
more involved something like this becomes, at least in terms of the number of 
tools that have to be modified, the less likely anyone is going to be willing 
to attempt any of it.  Particularly if they're not coming from a mainframe 
background.

I recently came across a question in an online forum where the poster was 
asking how to tell if the input their program was reading was base64 encoded.  
The various responders were suggesting all sorts of things they could check to 
get some idea if it might be or not.  My suggestion would have been "try to 
decode it and see if it works.  If it does, it was base64 encoded."  I suspect 
something similar is going to wind up having to be done here.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[WF Konynenberg]

On 05/23/2016 01:22 AM, Mark Post wrote:

On 5/22/2016 at 10:45 AM, WF Konynenberg <w...@konynenberg.org> wrote:

In Linux/Unix, a file is simply a byte stream.  Any format information
about a structured file should be accessible to the application by some
means, so it can know how to parse the file. Since the CMS file system
stores structured files of different types/modes, this meta information
in the file system should somehow be made available to applications in
Linux/Unix when accessing the file system through cms-fuse.  There are

Sounds like you're starting to argue Alan's case here.  Have the file system 
driver do the heavy lifting for you.  I don't think that's likely to happen.


I'm not so sure.

I'm not suggesting that cmsfs-fuse actually *handle* the various CMS
file formats ("do the heavy lifting"), convert the format, and somehow
magically make the files appear as some comparable common Linux file
format to Linux programs.
I'm just saying that any information that's available in the CMS file
system "out of band" that is somehow relevant to an application trying
to read/write any given file must be made available to the Linux
programs in some way or other.  By simply dropping such additional info
on the floor, it may become rather difficult for Linux programs to
correctly interpret the raw file data presented by cmfsf-fuse, and it
may be impossible to create a new file with correct meta data.

Cmsfs-fuse could provide this out-of-band info in various ways. There's
no "perfect" solution.  It just somehow needs to make this info
available to the Linux program.  I'm guessing that cmsfs-fuse currently
already somehow "grabs" the raw record data from the CMS files, in
whatever file format it happens to be, and presents this raw data in
Linux as a stream of bytes with some minimal logical structure.  It
doesn't need to do anything smarter than that.  It should just make sure
that if there is "out of band" info in the CMS FS describing the record
format and other details, this info is somehow presented to the Linux
programs for processing.

Note that many (binary) file formats in Linux contain some of this "meta
data" "in band", i.e. as part of a header in the file itself, which
describes relevant details of the file format, thus allowing the program
to correctly interpret the rest of the file.  The actual data part of
the file might then be structured as blocks, records, or whatever, and
it is up to the program to correctly handle this.




various ways to achieve that, e.g. represent each CMS file by 2 files in
Linux/Unix (similar, I think, to how "resource forks" are handled in
Apple systems), one containing the data, one containing the meta data,
or, alternatively, add a header to each file that represents the meta
data from the file system (though this might break some existing
applications that currently expect to find only the actual data in the
file and somehow "magically" know the meta data...), etc, etc. Finding
the most suitable model will likely require some creative thought on the
part of the developers.  Perhaps the representation model can be made to
depend on a mount option, with the default being backward compatible
lack of metadata.

Just simply saying "sorry, this critical meta data from the CMS file
system is not available in Linux" basically means unnecessarily
crippling Linux access to the CMS file system.

Again, Alan's original suggestion was to teach cmsfs-fuse about such things.

I suppose one could use a combination of cmsfs-fuse and Rick Troth's wonderful 
CMSFS package.

s390vsl204:~ # cmsfs-fuse /dev/dasdg /mnt
s390vsl204:~ # l /mnt
total 34024
drwxrwx--- 2 root root  512 May  4 18:56 ./
drwxr-xr-x 1 root root  172 May 18 19:00 ../
-rw-rw 1 root root  160 May  4 17:07 FTPBOOT.PARM
-rw-rw 1 root root 34812720 May  4 18:56 INITRAMF.IMG
-rw-rw 1 root root 2000 Oct  1  2012 KAKKA2.EXEC
-rw-rw 1 root root 2000 Oct  1  2012 KAKKA3.EXEC
-rw-rw 1 root root 2000 Oct  1  2012 LINUX.EXEC
-rw-rw 1 root root 2000 Oct  1  2012 S11S2GM.EXEC

s390vsl204:~ # cmsfslst -f /dev/dasdg
FILENAME FILETYPE FM FORMAT LRECL   RECS BLOCKS DATE TIME
  DIRECTOR P0 F 64  8  1  5/04/2016 19:56:38
  ALLOCMAP P0 F   4096  2  2  5/04/2016 19:56:38
FTPBOOT  PARM B1 F 80  2  1  5/04/2016 18:07:40
S11S2GM  EXEC Z1 F 80 25  1 10/01/2012  6:23:36
KAKKA2   EXEC Z1 F 80 25  1 10/01/2012  6:27:42
KAKKA3   EXEC Z1 F 80 25  1 10/01/2012  6:28:18
LINUXEXEC Z1 F 80 25  1 10/01/2012  6:27:00
INITRAMF IMG  B1 F 80 435159   8500  5/04/2016 19:56:38

I don't think the file command is going to be extended to handle this, howeve

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Mark Post]

>>> On 5/22/2016 at 10:45 AM, WF Konynenberg <w...@konynenberg.org> wrote: 
> In Linux/Unix, a file is simply a byte stream.  Any format information
> about a structured file should be accessible to the application by some
> means, so it can know how to parse the file. Since the CMS file system
> stores structured files of different types/modes, this meta information
> in the file system should somehow be made available to applications in
> Linux/Unix when accessing the file system through cms-fuse.  There are

Sounds like you're starting to argue Alan's case here.  Have the file system 
driver do the heavy lifting for you.  I don't think that's likely to happen.

> various ways to achieve that, e.g. represent each CMS file by 2 files in
> Linux/Unix (similar, I think, to how "resource forks" are handled in
> Apple systems), one containing the data, one containing the meta data,
> or, alternatively, add a header to each file that represents the meta
> data from the file system (though this might break some existing
> applications that currently expect to find only the actual data in the
> file and somehow "magically" know the meta data...), etc, etc. Finding
> the most suitable model will likely require some creative thought on the
> part of the developers.  Perhaps the representation model can be made to
> depend on a mount option, with the default being backward compatible
> lack of metadata.
> 
> Just simply saying "sorry, this critical meta data from the CMS file
> system is not available in Linux" basically means unnecessarily
> crippling Linux access to the CMS file system.

Again, Alan's original suggestion was to teach cmsfs-fuse about such things.

I suppose one could use a combination of cmsfs-fuse and Rick Troth's wonderful 
CMSFS package.

s390vsl204:~ # cmsfs-fuse /dev/dasdg /mnt
s390vsl204:~ # l /mnt
total 34024
drwxrwx--- 2 root root  512 May  4 18:56 ./
drwxr-xr-x 1 root root  172 May 18 19:00 ../
-rw-rw 1 root root  160 May  4 17:07 FTPBOOT.PARM
-rw-rw 1 root root 34812720 May  4 18:56 INITRAMF.IMG
-rw-rw 1 root root 2000 Oct  1  2012 KAKKA2.EXEC
-rw-rw 1 root root 2000 Oct  1  2012 KAKKA3.EXEC
-rw-rw 1 root root 2000 Oct  1  2012 LINUX.EXEC
-rw-rw 1 root root 2000 Oct  1  2012 S11S2GM.EXEC

s390vsl204:~ # cmsfslst -f /dev/dasdg
FILENAME FILETYPE FM FORMAT LRECL   RECS BLOCKS DATE TIME
 DIRECTOR P0 F 64  8  1  5/04/2016 19:56:38
 ALLOCMAP P0 F   4096  2  2  5/04/2016 19:56:38
FTPBOOT  PARM B1 F 80  2  1  5/04/2016 18:07:40
S11S2GM  EXEC Z1 F 80 25  1 10/01/2012  6:23:36
KAKKA2   EXEC Z1 F 80 25  1 10/01/2012  6:27:42
KAKKA3   EXEC Z1 F 80 25  1 10/01/2012  6:28:18
LINUXEXEC Z1 F 80 25  1 10/01/2012  6:27:00
INITRAMF IMG  B1 F 80 435159   8500  5/04/2016 19:56:38

I don't think the file command is going to be extended to handle this, however.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Mark Post]

>>> On 5/22/2016 at 11:02 AM, Doug Shupe  wrote: 
> How about "Listfile FN FT FM" to gather the simple file information?

That would have to be done is CMS, since there is no such concept (or program) 
in Linux.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Doug Shupe]

How about "Listfile FN FT FM" to gather the simple file information?
Cheers

On May 22, 2016, at 10:45, WF Konynenberg  wrote:

> On 05/22/2016 03:28 PM, Alan Altmark wrote:
> On Friday, 05/20/2016 at 03:05 GMT, Hendrik Brueckner
>  wrote:
>> To recognize the file format the "magic" patterns are used.  See also
> the
>> definitions in /usr/share/misc/magic and magic(5) man page.  If you know
>> how to detect packed CMS files, you can add the signature there (or for
>> testing locally) and check if file is then able to recognize it.
> Unfortunately part of the signature is in the LRECL 1024 and RECFM F, but
> those aren't visible to apps.  The only thing you can do in Linux (without
> actually unpacking the file) is
> - Check for a file length that's a multiple of 1024
> - The first four bytes are 0x000140E5 or 0x000140C6

Hmm,

That doesn't seem right.
In Linux/Unix, a file is simply a byte stream.  Any format information
about a structured file should be accessible to the application by some
means, so it can know how to parse the file. Since the CMS file system
stores structured files of different types/modes, this meta information
in the file system should somehow be made available to applications in
Linux/Unix when accessing the file system through cms-fuse.  There are
various ways to achieve that, e.g. represent each CMS file by 2 files in
Linux/Unix (similar, I think, to how "resource forks" are handled in
Apple systems), one containing the data, one containing the meta data,
or, alternatively, add a header to each file that represents the meta
data from the file system (though this might break some existing
applications that currently expect to find only the actual data in the
file and somehow "magically" know the meta data...), etc, etc. Finding
the most suitable model will likely require some creative thought on the
part of the developers.  Perhaps the representation model can be made to
depend on a mount option, with the default being backward compatible
lack of metadata.

Just simply saying "sorry, this critical meta data from the CMS file
system is not available in Linux" basically means unnecessarily
crippling Linux access to the CMS file system.


Willy

> 
> Alan Altmark
> 
> Senior Managing z/VM and Linux Consultant
> Lab Services System z Delivery Practice
> IBM Systems & Technology Group
> ibm.com/systems/services/labservices
> office: 607.429.3323
> mobile; 607.321.7556
> alan_altm...@us.ibm.com
> IBM Endicott
> 
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[WF Konynenberg]

On 05/22/2016 07:12 AM, Alan Altmark wrote:

On Saturday, 05/21/2016 at 02:00 GMT, WF Konynenberg 
wrote:


That means the "native" way to deal with these on Linux would be
to implement some "cmspack" utility which can pack/unpack such files (as
was indicated, the assembler source code is available so it's easy to
figure out how to do this and reimplement it in a bit of C code), and
teach the file command about this file format so it can correctly report

it.

If folks think having a separate utility and teaching Linux folks to
recognize a packed CMS file when they see it is the right way to go, then
I sure won't stand in the way!



I think the rule should probably be that
- if the "packed" status of a file has always been fully transparent to
CMS applications and users, and handled completely within the file
system code of the operating system, then it would make sense to
implement this as part of the cms-fuse file system, similar to how some
Linux filesystems can transparently compress files internally without
exposing this to applications,
but
- if the "packed" status of a CMS file is user visible to some extent
and requires some explicit handling, e.g. applications or users being
aware that they need to unpack before they can use the data, etc, then
it should probably be handled the same way that the various other
compressed file formats are handled in Linux/Unix, i.e. make sure
users/applications can identify these formats, and have a set of
commands to compress/uncompress the files, which can, if desired, be
used in a pipeline to do so "on the fly".


Willy

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[WF Konynenberg]

On 05/22/2016 03:28 PM, Alan Altmark wrote:

On Friday, 05/20/2016 at 03:05 GMT, Hendrik Brueckner
 wrote:

To recognize the file format the "magic" patterns are used.  See also

the

definitions in /usr/share/misc/magic and magic(5) man page.  If you know
how to detect packed CMS files, you can add the signature there (or for
testing locally) and check if file is then able to recognize it.

Unfortunately part of the signature is in the LRECL 1024 and RECFM F, but
those aren't visible to apps.  The only thing you can do in Linux (without
actually unpacking the file) is
- Check for a file length that's a multiple of 1024
- The first four bytes are 0x000140E5 or 0x000140C6


Hmm,

That doesn't seem right.
In Linux/Unix, a file is simply a byte stream.  Any format information
about a structured file should be accessible to the application by some
means, so it can know how to parse the file. Since the CMS file system
stores structured files of different types/modes, this meta information
in the file system should somehow be made available to applications in
Linux/Unix when accessing the file system through cms-fuse.  There are
various ways to achieve that, e.g. represent each CMS file by 2 files in
Linux/Unix (similar, I think, to how "resource forks" are handled in
Apple systems), one containing the data, one containing the meta data,
or, alternatively, add a header to each file that represents the meta
data from the file system (though this might break some existing
applications that currently expect to find only the actual data in the
file and somehow "magically" know the meta data...), etc, etc. Finding
the most suitable model will likely require some creative thought on the
part of the developers.  Perhaps the representation model can be made to
depend on a mount option, with the default being backward compatible
lack of metadata.

Just simply saying "sorry, this critical meta data from the CMS file
system is not available in Linux" basically means unnecessarily
crippling Linux access to the CMS file system.


Willy



Alan Altmark

Senior Managing z/VM and Linux Consultant
Lab Services System z Delivery Practice
IBM Systems & Technology Group
ibm.com/systems/services/labservices
office: 607.429.3323
mobile; 607.321.7556
alan_altm...@us.ibm.com
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/



--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Alan Altmark]

On Friday, 05/20/2016 at 03:05 GMT, Hendrik Brueckner 
 wrote:
> To recognize the file format the "magic" patterns are used.  See also 
the
> definitions in /usr/share/misc/magic and magic(5) man page.  If you know
> how to detect packed CMS files, you can add the signature there (or for
> testing locally) and check if file is then able to recognize it.

Unfortunately part of the signature is in the LRECL 1024 and RECFM F, but 
those aren't visible to apps.  The only thing you can do in Linux (without 
actually unpacking the file) is
- Check for a file length that's a multiple of 1024
- The first four bytes are 0x000140E5 or 0x000140C6

Alan Altmark

Senior Managing z/VM and Linux Consultant
Lab Services System z Delivery Practice
IBM Systems & Technology Group
ibm.com/systems/services/labservices
office: 607.429.3323
mobile; 607.321.7556
alan_altm...@us.ibm.com
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Alan Altmark]

On Saturday, 05/21/2016 at 02:00 GMT, WF Konynenberg  
wrote:
> Aye, from what I have read about these packed files here, it seems to me
> to be much like the various compressed file formats in use on Linux/Unix
> (.z, .gz, .bz, .xz, etc), except that perhaps the CMS pack tool neglects
> to modify the file name extension to indicate that this is now a packed
> file. 

There is no 'neglect'.  Packed-ness is not tied to the filetype.

> That means the "native" way to deal with these on Linux would be
> to implement some "cmspack" utility which can pack/unpack such files (as
> was indicated, the assembler source code is available so it's easy to
> figure out how to do this and reimplement it in a bit of C code), and
> teach the file command about this file format so it can correctly report 
it.

If folks think having a separate utility and teaching Linux folks to 
recognize a packed CMS file when they see it is the right way to go, then 
I sure won't stand in the way!

Alan Altmark

Senior Managing z/VM and Linux Consultant
Lab Services System z Delivery Practice
IBM Systems & Technology Group
ibm.com/systems/services/labservices
office: 607.429.3323
mobile; 607.321.7556
alan_altm...@us.ibm.com
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[WF Konynenberg]

Aye, from what I have read about these packed files here, it seems to me 
to be much like the various compressed file formats in use on Linux/Unix 
(.z, .gz, .bz, .xz, etc), except that perhaps the CMS pack tool neglects 
to modify the file name extension to indicate that this is now a packed 
file.  That means the "native" way to deal with these on Linux would be 
to implement some "cmspack" utility which can pack/unpack such files (as 
was indicated, the assembler source code is available so it's easy to 
figure out how to do this and reimplement it in a bit of C code), and 
teach the file command about this file format so it can correctly report it.


Trying to "hide" this by having the cms-fuse file system somehow 
"transparently" do the unpacking on these files seems to me to be the 
wrong approach and not fitting the Linux philosophy.  So yes, I would 
hope the cms-fuse developers would push back on such an idea. Linux/Unix 
does not have a notion of "file type" in the operating system.  All 
files are a stream of bytes, and if you open a file, you get that exact 
stream of bytes.  If said stream of bytes happens to be in some 
compressed format, you are expected to handle this at the 
user/application level, e.g. by opening the file with the suitable 
uncompress tool, and piping the (uncompressed) output of that to the 
program that you want to process the uncompressed file data.



Willy

On 05/19/2016 10:41 PM, Neale Ferguson wrote:

We could teach the “file" command on linux to recognize such files. It
just uses signature-type information.

On 5/19/16, 4:27 PM, "Linux on 390 Port on behalf of Scott Rohling"
 wrote:


If I 'browse' (or TYPE for that matter)  a COPYFILE packed file - it looks
much the same as it did from the Linux...

That being said - if I was the cms-fuse developer I would probably push
back on this...   just because XEDIT was smart and did the unpack
internally doesn't mean the driver should try and do the same.   Isn't the
onus on the application opening the file to read it correctly - just as it
is in under CMS?Is cms-fuse supposed to incorporate the unpack?  What
about terse, vmarc, et al?Seems like a slippery slope unless I just
plain misunderstood you.  (never happens :-)

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/



--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Tito Garrido]

Hi Mauro,

To unpack I had to go z/VM and "COPYFILE / (unpack oldd" on each file.

Regards,

Tito


On Thu, May 19, 2016 at 5:24 PM, Mauro Souza  wrote:

> That was a question I had for some time but never got time to pursue: how
> to unpack the files? It's something from Linux or from zVM?
> On May 19, 2016 17:02, "Alan Altmark"  wrote:
>
> > On Thursday, 05/19/2016 at 06:28 GMT, Tito Garrido <
> titogarr...@gmail.com>
> > wrote:
> > > Issue solved by Alan, the files were packed by VMLOGS... I had unpacked
> > the
> > > file and Linux can read it.
> >
> > I would probably open a ticket on the cms-fuse support.  Alas, there's no
> > indicator of "packed-ness" in the file descriptor (FST), so cms-fuse
> would
> > have to heuristically determine the packed/unpacked status of a file.
> > (That's how XEDIT works.)
> >
> > Alan Altmark
> >
> > Senior Managing z/VM and Linux Consultant
> > Lab Services System z Delivery Practice
> > IBM Systems & Technology Group
> > ibm.com/systems/services/labservices
> > office: 607.429.3323
> > mobile; 607.321.7556
> > alan_altm...@us.ibm.com
> > IBM Endicott
> >
> > --
> > For LINUX-390 subscribe / signoff / archive access instructions,
> > send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> > visit
> > http://www.marist.edu/htbin/wlvindex?LINUX-390
> > --
> > For more information on Linux on System z, visit
> > http://wiki.linuxvm.org/
> >
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>



-- 

Linux User #387870
.
 _/_õ|__|
..º[ .-.___.-._| . . . .
.__( o)__( o).:___

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Scott Rohling]

Thanks for that pointer Hendrik ...   it sounds very appealing on a Friday
to dabble in magic!

Scott Rohling



On Fri, May 20, 2016 at 8:03 AM, Hendrik Brueckner <
brueck...@linux.vnet.ibm.com> wrote:

> On Fri, May 20, 2016 at 07:53:22AM -0700, Scott Rohling wrote:
> > My point was that what you see in Linux and CMS are not two different
> > things..   a Linux 'cat' and CMS 'TYPE' are pretty equivalent -- and a
> > packed file looks garbled using either one.
> >
> > I think Neale had a good idea in making the Linux 'file' command
> recognize
> > this particular signature so at least Linux knows it is packed and not
> text
> > format - just as it knows about tar, gzip, etc even without a particular
> > filetype in the name.
>
> To recognize the file format the "magic" patterns are used.  See also the
> definitions in /usr/share/misc/magic and magic(5) man page.  If you know
> how to detect packed CMS files, you can add the signature there (or for
> testing locally) and check if file is then able to recognize it.
>
> Kind regards,
>   Hendrik
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Hendrik Brueckner]

On Fri, May 20, 2016 at 07:53:22AM -0700, Scott Rohling wrote:
> My point was that what you see in Linux and CMS are not two different
> things..   a Linux 'cat' and CMS 'TYPE' are pretty equivalent -- and a
> packed file looks garbled using either one.
>
> I think Neale had a good idea in making the Linux 'file' command recognize
> this particular signature so at least Linux knows it is packed and not text
> format - just as it knows about tar, gzip, etc even without a particular
> filetype in the name.

To recognize the file format the "magic" patterns are used.  See also the
definitions in /usr/share/misc/magic and magic(5) man page.  If you know
how to detect packed CMS files, you can add the signature there (or for
testing locally) and check if file is then able to recognize it.

Kind regards,
  Hendrik

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Scott Rohling]

My point was that what you see in Linux and CMS are not two different
things..   a Linux 'cat' and CMS 'TYPE' are pretty equivalent -- and a
packed file looks garbled using either one.

I think Neale had a good idea in making the Linux 'file' command recognize
this particular signature so at least Linux knows it is packed and not text
format - just as it knows about tar, gzip, etc even without a particular
filetype in the name.

Scott Rohling

On Fri, May 20, 2016 at 7:16 AM, Alan Altmark 
wrote:

> On Thursday, 05/19/2016 at 08:28 GMT, Scott Rohling
>  wrote:
> > If I 'browse' (or TYPE for that matter)  a COPYFILE packed file - it
> looks
> > much the same as it did from the Linux...
> >
> > That being said - if I was the cms-fuse developer I would probably push
> > back on this...   just because XEDIT was smart and did the unpack
> > internally doesn't mean the driver should try and do the same.   Isn't
> the
> > onus on the application opening the file to read it correctly - just as
> it
> > is in under CMS?Is cms-fuse supposed to incorporate the unpack? What
> > about terse, vmarc, et al?Seems like a slippery slope unless I just
> > plain misunderstood you.  (never happens :-)
>
> No, it's not up to the application.  In general, YOU are required to
> unpack any file you want some application to read.  XEDIT is a notable
> exception.
>
> If you install the HSX package from the VM Download Library, it will
> unpack (COPYFILE, TERSE, FCOPY) files in real-time, making their use
> transparent to the application.  It would be nice if cms-fuse could deal
> them.  Or one of them thar fancy shmancy Linux script thingies ought do be
> able to expand the file onto an in-memory disk.
>
> But if it weren't for the ubiquity of packed files, I probably wouldn't be
> all that concerned about it.  I just don't like that what you see in CMS
> and what you see in Linux are two different things.  People not steeped in
> CMS file system lore aren't going to grok a packed file, *especially*
> since it's not a visible file attribute.  (A few times I've thought about
> trying to to make it one.)
>
> I guess I should put a blurb on my web page about the format of a packed
> file.
>
> Alan Altmark
>
> Senior Managing z/VM and Linux Consultant
> Lab Services System z Delivery Practice
> IBM Systems & Technology Group
> ibm.com/systems/services/labservices
> office: 607.429.3323
> mobile; 607.321.7556
> alan_altm...@us.ibm.com
> IBM Endicott
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Alan Altmark]

On Thursday, 05/19/2016 at 08:28 GMT, Scott Rohling 
 wrote:
> If I 'browse' (or TYPE for that matter)  a COPYFILE packed file - it 
looks
> much the same as it did from the Linux...
>
> That being said - if I was the cms-fuse developer I would probably push
> back on this...   just because XEDIT was smart and did the unpack
> internally doesn't mean the driver should try and do the same.   Isn't 
the
> onus on the application opening the file to read it correctly - just as 
it
> is in under CMS?Is cms-fuse supposed to incorporate the unpack? What
> about terse, vmarc, et al?Seems like a slippery slope unless I just
> plain misunderstood you.  (never happens :-)

No, it's not up to the application.  In general, YOU are required to 
unpack any file you want some application to read.  XEDIT is a notable 
exception.

If you install the HSX package from the VM Download Library, it will 
unpack (COPYFILE, TERSE, FCOPY) files in real-time, making their use 
transparent to the application.  It would be nice if cms-fuse could deal 
them.  Or one of them thar fancy shmancy Linux script thingies ought do be 
able to expand the file onto an in-memory disk.

But if it weren't for the ubiquity of packed files, I probably wouldn't be 
all that concerned about it.  I just don't like that what you see in CMS 
and what you see in Linux are two different things.  People not steeped in 
CMS file system lore aren't going to grok a packed file, *especially* 
since it's not a visible file attribute.  (A few times I've thought about 
trying to to make it one.)

I guess I should put a blurb on my web page about the format of a packed 
file.

Alan Altmark

Senior Managing z/VM and Linux Consultant
Lab Services System z Delivery Practice
IBM Systems & Technology Group
ibm.com/systems/services/labservices
office: 607.429.3323
mobile; 607.321.7556
alan_altm...@us.ibm.com
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Doug Shupe]

Thank You Alan!
Had wanted to figure that out for a long time but never had time to do all the 
research. 
Best Regards,
Doug

On May 19, 2016, at 16:42, Alan Altmark  wrote:

On Thursday, 05/19/2016 at 08:24 GMT, Mauro Souza  
wrote:
> That was a question I had for some time but never got time to pursue:
how
> to unpack the files? It's something from Linux or from zVM?

Without an update to cms-fuse, it's simplest just to unpack them with CMS 
before you mount the disk.

There's nothing magical about packed files.  Those of you who have loaded 
the VM source (and who can read assembler) can see exactly how you expand 
a packed by by looking at label DMSXINPK in module DMSXIN ASSEMBLE.

Alan Altmark

Senior Managing z/VM and Linux Consultant
Lab Services System z Delivery Practice
IBM Systems & Technology Group
ibm.com/systems/services/labservices
office: 607.429.3323
mobile; 607.321.7556
alan_altm...@us.ibm.com
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Alan Altmark]

On Thursday, 05/19/2016 at 08:24 GMT, Mauro Souza  
wrote:
> That was a question I had for some time but never got time to pursue: 
how
> to unpack the files? It's something from Linux or from zVM?

Without an update to cms-fuse, it's simplest just to unpack them with CMS 
before you mount the disk.

There's nothing magical about packed files.  Those of you who have loaded 
the VM source (and who can read assembler) can see exactly how you expand 
a packed by by looking at label DMSXINPK in module DMSXIN ASSEMBLE.

Alan Altmark

Senior Managing z/VM and Linux Consultant
Lab Services System z Delivery Practice
IBM Systems & Technology Group
ibm.com/systems/services/labservices
office: 607.429.3323
mobile; 607.321.7556
alan_altm...@us.ibm.com
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Neale Ferguson]

We could teach the “file" command on linux to recognize such files. It
just uses signature-type information.

On 5/19/16, 4:27 PM, "Linux on 390 Port on behalf of Scott Rohling"
 wrote:

>If I 'browse' (or TYPE for that matter)  a COPYFILE packed file - it looks
>much the same as it did from the Linux...
>
>That being said - if I was the cms-fuse developer I would probably push
>back on this...   just because XEDIT was smart and did the unpack
>internally doesn't mean the driver should try and do the same.   Isn't the
>onus on the application opening the file to read it correctly - just as it
>is in under CMS?Is cms-fuse supposed to incorporate the unpack?  What
>about terse, vmarc, et al?Seems like a slippery slope unless I just
>plain misunderstood you.  (never happens :-)

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Scott Rohling]

If I 'browse' (or TYPE for that matter)  a COPYFILE packed file - it looks
much the same as it did from the Linux...

That being said - if I was the cms-fuse developer I would probably push
back on this...   just because XEDIT was smart and did the unpack
internally doesn't mean the driver should try and do the same.   Isn't the
onus on the application opening the file to read it correctly - just as it
is in under CMS?Is cms-fuse supposed to incorporate the unpack?  What
about terse, vmarc, et al?Seems like a slippery slope unless I just
plain misunderstood you.  (never happens :-)

Scott Rohling



On Thu, May 19, 2016 at 1:00 PM, Alan Altmark 
wrote:

> On Thursday, 05/19/2016 at 06:28 GMT, Tito Garrido 
> wrote:
> > Issue solved by Alan, the files were packed by VMLOGS... I had unpacked
> the
> > file and Linux can read it.
>
> I would probably open a ticket on the cms-fuse support.  Alas, there's no
> indicator of "packed-ness" in the file descriptor (FST), so cms-fuse would
> have to heuristically determine the packed/unpacked status of a file.
> (That's how XEDIT works.)
>
> Alan Altmark
>
> Senior Managing z/VM and Linux Consultant
> Lab Services System z Delivery Practice
> IBM Systems & Technology Group
> ibm.com/systems/services/labservices
> office: 607.429.3323
> mobile; 607.321.7556
> alan_altm...@us.ibm.com
> IBM Endicott
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Mauro Souza]

That was a question I had for some time but never got time to pursue: how
to unpack the files? It's something from Linux or from zVM?
On May 19, 2016 17:02, "Alan Altmark"  wrote:

> On Thursday, 05/19/2016 at 06:28 GMT, Tito Garrido 
> wrote:
> > Issue solved by Alan, the files were packed by VMLOGS... I had unpacked
> the
> > file and Linux can read it.
>
> I would probably open a ticket on the cms-fuse support.  Alas, there's no
> indicator of "packed-ness" in the file descriptor (FST), so cms-fuse would
> have to heuristically determine the packed/unpacked status of a file.
> (That's how XEDIT works.)
>
> Alan Altmark
>
> Senior Managing z/VM and Linux Consultant
> Lab Services System z Delivery Practice
> IBM Systems & Technology Group
> ibm.com/systems/services/labservices
> office: 607.429.3323
> mobile; 607.321.7556
> alan_altm...@us.ibm.com
> IBM Endicott
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Alan Altmark]

On Thursday, 05/19/2016 at 06:28 GMT, Tito Garrido  
wrote:
> Issue solved by Alan, the files were packed by VMLOGS... I had unpacked 
the
> file and Linux can read it.

I would probably open a ticket on the cms-fuse support.  Alas, there's no 
indicator of "packed-ness" in the file descriptor (FST), so cms-fuse would 
have to heuristically determine the packed/unpacked status of a file. 
(That's how XEDIT works.)

Alan Altmark

Senior Managing z/VM and Linux Consultant
Lab Services System z Delivery Practice
IBM Systems & Technology Group
ibm.com/systems/services/labservices
office: 607.429.3323
mobile; 607.321.7556
alan_altm...@us.ibm.com
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Reading VMLOGS files from Linux (cmsfs-fuse)

[Tito Garrido]

Issue solved by Alan, the files were packed by VMLOGS... I had unpacked the
file and Linux can read it.

Thanks!

Tito

On Wed, May 18, 2016 at 4:36 PM, Tito Garrido <titogarr...@gmail.com> wrote:

> Hi Folks,
>
> I have a bunch of console logs from VMLOGS that I would like to read from
> Linux. I have used cmsfs-fuse -a to mount it on /mnt but there are a lot of
> strange chars in the file:
>
>  cat MAINT9.SYSPROG
>  FdoDMSACP723I T (1193) R/O%nDMSACP723I B (5E5) R/O_nDMSACP723I D (51D)
> R/O_nDMSACP723I E (551) R/O_▒coPRESS ENTER TO CONTINUE%▒c▒Ready;
> T=0.01/0.01 14:26:05▒ebkrjob▒'▒Ready; T=0.46/0.48
> 14:29:22▒elogoff▒'▒CONNECT= 00:03:23 VIRTCPU= 000:00.46 TOTCPU=
> 000:00.49(vLOGOFF AT 14:29:26 BRT FRIDAY 05/13/16)▒
>
> Is there a way to fix it?
>
> Regards,
>
> Tito
> --
>
> Linux User #387870
> .
>  _/_õ|__|
> ..º[ .-.___.-._| . . . .
> .__( o)__( o).:___
>



-- 

Linux User #387870
.
 _/_õ|__|
..º[ .-.___.-._| . . . .
.__( o)__( o).:___

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Reading VMLOGS files from Linux (cmsfs-fuse)

[Tito Garrido]

Hi Folks,

I have a bunch of console logs from VMLOGS that I would like to read from
Linux. I have used cmsfs-fuse -a to mount it on /mnt but there are a lot of
strange chars in the file:

 cat MAINT9.SYSPROG
 FdoDMSACP723I T (1193) R/O%nDMSACP723I B (5E5) R/O_nDMSACP723I D (51D)
R/O_nDMSACP723I E (551) R/O_▒coPRESS ENTER TO CONTINUE%▒c▒Ready;
T=0.01/0.01 14:26:05▒ebkrjob▒'▒Ready; T=0.46/0.48
14:29:22▒elogoff▒'▒CONNECT= 00:03:23 VIRTCPU= 000:00.46 TOTCPU=
000:00.49(vLOGOFF AT 14:29:26 BRT FRIDAY 05/13/16)▒

Is there a way to fix it?

Regards,

Tito
-- 

Linux User #387870
.
 _/_õ|__|
..º[ .-.___.-._| . . . .
.__( o)__( o).:___

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Cmsfs

[Mark Post]

>>> On 3/3/2016 at 08:21 AM, Michael Weiner  wrote: 
> That worked. But I usually mount with -a (for the text)
> 
> How can I do both?

By doing "-a -o allow_other" ?


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Cmsfs

[Michael Weiner]

Got it!!!

THank you!!!

On Thu, Mar 3, 2016 at 9:21 AM, Michael Weiner <mwei...@infinite-blue.com>
wrote:

> Mark
>
> That worked. But I usually mount with -a (for the text)
>
> How can I do both?
>
>
>
> On Thu, Mar 3, 2016 at 9:08 AM, Mark Post <mp...@suse.com> wrote:
>
>> >>> On 3/3/2016 at 07:49 AM, Michael Weiner <mwei...@infinite-blue.com>
>> wrote:
>> > Interesting, I just removed test and mounted my CMS disk to htdocs and
>> it
>> > is still complaining about the permissions
>>
>> I believe what you're looking for is "-o allow_other" on the cmsfs-fuse
>> command.
>>
>>
>> Mark Post
>>
>> --
>> For LINUX-390 subscribe / signoff / archive access instructions,
>> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
>> visit
>> http://www.marist.edu/htbin/wlvindex?LINUX-390
>> --
>> For more information on Linux on System z, visit
>> http://wiki.linuxvm.org/
>>
>
>
>
> --
> Michael Weiner
> Systems Admin
> Infinity Systems Software, Inc.
> One Penn Plaza Suite 2010
> New York, NY 10119
> o: (646) 405-9300
> c: (845) 641-0517
>
>
>


--
Michael Weiner
Systems Admin
Infinity Systems Software, Inc.
One Penn Plaza Suite 2010
New York, NY 10119
o: (646) 405-9300
c: (845) 641-0517

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Cmsfs

[Michael Weiner]

Mark

That worked. But I usually mount with -a (for the text)

How can I do both?



On Thu, Mar 3, 2016 at 9:08 AM, Mark Post <mp...@suse.com> wrote:

> >>> On 3/3/2016 at 07:49 AM, Michael Weiner <mwei...@infinite-blue.com>
> wrote:
> > Interesting, I just removed test and mounted my CMS disk to htdocs and it
> > is still complaining about the permissions
>
> I believe what you're looking for is "-o allow_other" on the cmsfs-fuse
> command.
>
>
> Mark Post
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>



--
Michael Weiner
Systems Admin
Infinity Systems Software, Inc.
One Penn Plaza Suite 2010
New York, NY 10119
o: (646) 405-9300
c: (845) 641-0517

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Cmsfs

[Mark Post]

>>> On 3/3/2016 at 07:49 AM, Michael Weiner <mwei...@infinite-blue.com> wrote: 
> Interesting, I just removed test and mounted my CMS disk to htdocs and it
> is still complaining about the permissions

I believe what you're looking for is "-o allow_other" on the cmsfs-fuse command.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Cmsfs

[Michael MacIsaac]

Mike,

Apache runs as the User and Group defined in /etc/apache2/uid.conf, the
group 'www' by default

Maybe try this:
-) Perhaps rename /srv/www/htdocs/test to something like
/srv/www/htdocs/cmsfs
-) Make 'www' the group owner of that directory: chgrp www cmsfs
-) Mount the CMS file system over cmsfs/ with csmfs-fuse
-) service apache2 restart (may not be needed but won't hurt)

Then try again http://your.server/cmsfs/MICHAELW.PDF

-Mike

On Thu, Mar 3, 2016 at 8:49 AM, Michael Weiner <mwei...@infinite-blue.com>
wrote:

> Interesting, I just removed test and mounted my CMS disk to htdocs and it
> is still complaining about the permissions
>
> (because search permissions are missing on a component of the path)
>
> The permissions of the files are
>
> -rw-rw 1 root root 293448 Mar  2 16:48 MICHAELW.PDF
>
> I copied MICHAELW.PDF ../
>
> umounted htdocs
>
> chmod 644 MICHAELW.PDF and moved it back to htdocs and it works
>
> Still doesn't work when I mount the CMS DISK
>
>
> On Thu, Mar 3, 2016 at 8:43 AM, Michael MacIsaac <mike99...@gmail.com>
> wrote:
>
> > Mike,
> >
> > Either add a specific directory section for /srv/www/htdocs/test or it
> will
> > inherit from /srv/www/htdocs which is defined in
> > /etc/apache2/default-server.conf.
> >
> > Who knows, maybe it's working now. What do you get when you specify a URL
> > such as http://your.server/test/MICHAELW.PDF
> >
> > If you get an error, look at tail of the updated log files in
> > /var/log/apache2
> >
> > HTH
> >
> > -Mike
> >
> > On Thu, Mar 3, 2016 at 8:21 AM, Michael Weiner <
> mwei...@infinite-blue.com>
> > wrote:
> >
> > > No I don't believe that I do.
> > >
> > > Do I need to add?
> > >
> > > Sent from my iPhone
> > >
> > > > On Mar 3, 2016, at 8:13 AM, Michael MacIsaac <mike99...@gmail.com>
> > > wrote:
> > > >
> > > > Mike,
> > > >
> > > >> I want to set these permissions so I can view the files using
> Apache.
> > > > Do you have a Directory section in the Apache conf file for
> > > > /srv/www/htdocs/test?
> > > >
> > > > If so does it have:
> > > >Order allow,deny
> > > >Allow from all
> > > >
> > > >-Mike M
> > > >
> > > > On Thu, Mar 3, 2016 at 8:02 AM, Michael Weiner <
> > > mwei...@infinite-blue.com>
> > > > wrote:
> > > >
> > > >> I tried setting the permissions of the folder.
> > > >>
> > > >> I want to set these permissions so I can view the files using
> Apache.
> > > >>
> > > >> Sent from my iPhone
> > > >>
> > > >>>> On Mar 3, 2016, at 7:47 AM, Michael MacIsaac <mike99...@gmail.com
> >
> > > >>> wrote:
> > > >>>
> > > >>> Mike,
> > > >>>
> > > >>> Why do you want to set permissions on CMS files?
> > > >>>
> > > >>>> Any suggestions?
> > > >>> Can you set the permissions of the mount point, and they will
> pertain
> > > to
> > > >>> the mounted CMS files?
> > > >>>
> > > >>>   -Mike M
> > > >>>
> > > >>> On Thu, Mar 3, 2016 at 7:32 AM, Michael Weiner <
> > > >> mwei...@infinite-blue.com>
> > > >>> wrote:
> > > >>>
> > > >>>> I think I posted my question in the wrong group.
> > > >>>>
> > > >>>> Scott, thanks for your help.
> > > >>>>
> > > >>>> According to the documentation the cmsfuse unmask command should
> be
> > > able
> > > >>>> to change the permissions. Two things I tried.
> > > >>>>
> > > >>>> sles12:/srv/www/htdocs/test # chmod 644 *
> > > >>>> chmod: changing permissions of âCMSTEST.EXECâ: Function not
> > > implemented
> > > >>>> chmod: changing permissions of âMICHAELW.PDFâ: Function not
> > > implemented
> > > >>>> chmod: changing permissions of âPROFILE.EXECâ: Function not
> > > implemented
> > > >>>> chmod: changing permissions of âPROFILE2.EXECâ: Function not
> > > implemented
> > > >>>>
> > > >>>> I also tried
> > > >>&

Re: Cmsfs

[Michael Weiner]

Interesting, I just removed test and mounted my CMS disk to htdocs and it
is still complaining about the permissions

(because search permissions are missing on a component of the path)

The permissions of the files are

-rw-rw 1 root root 293448 Mar  2 16:48 MICHAELW.PDF

I copied MICHAELW.PDF ../

umounted htdocs

chmod 644 MICHAELW.PDF and moved it back to htdocs and it works

Still doesn't work when I mount the CMS DISK


On Thu, Mar 3, 2016 at 8:43 AM, Michael MacIsaac <mike99...@gmail.com>
wrote:

> Mike,
>
> Either add a specific directory section for /srv/www/htdocs/test or it will
> inherit from /srv/www/htdocs which is defined in
> /etc/apache2/default-server.conf.
>
> Who knows, maybe it's working now. What do you get when you specify a URL
> such as http://your.server/test/MICHAELW.PDF
>
> If you get an error, look at tail of the updated log files in
> /var/log/apache2
>
> HTH
>
> -Mike
>
> On Thu, Mar 3, 2016 at 8:21 AM, Michael Weiner <mwei...@infinite-blue.com>
> wrote:
>
> > No I don't believe that I do.
> >
> > Do I need to add?
> >
> > Sent from my iPhone
> >
> > > On Mar 3, 2016, at 8:13 AM, Michael MacIsaac <mike99...@gmail.com>
> > wrote:
> > >
> > > Mike,
> > >
> > >> I want to set these permissions so I can view the files using Apache.
> > > Do you have a Directory section in the Apache conf file for
> > > /srv/www/htdocs/test?
> > >
> > > If so does it have:
> > >Order allow,deny
> > >Allow from all
> > >
> > >-Mike M
> > >
> > > On Thu, Mar 3, 2016 at 8:02 AM, Michael Weiner <
> > mwei...@infinite-blue.com>
> > > wrote:
> > >
> > >> I tried setting the permissions of the folder.
> > >>
> > >> I want to set these permissions so I can view the files using Apache.
> > >>
> > >> Sent from my iPhone
> > >>
> > >>>> On Mar 3, 2016, at 7:47 AM, Michael MacIsaac <mike99...@gmail.com>
> > >>> wrote:
> > >>>
> > >>> Mike,
> > >>>
> > >>> Why do you want to set permissions on CMS files?
> > >>>
> > >>>> Any suggestions?
> > >>> Can you set the permissions of the mount point, and they will pertain
> > to
> > >>> the mounted CMS files?
> > >>>
> > >>>   -Mike M
> > >>>
> > >>> On Thu, Mar 3, 2016 at 7:32 AM, Michael Weiner <
> > >> mwei...@infinite-blue.com>
> > >>> wrote:
> > >>>
> > >>>> I think I posted my question in the wrong group.
> > >>>>
> > >>>> Scott, thanks for your help.
> > >>>>
> > >>>> According to the documentation the cmsfuse unmask command should be
> > able
> > >>>> to change the permissions. Two things I tried.
> > >>>>
> > >>>> sles12:/srv/www/htdocs/test # chmod 644 *
> > >>>> chmod: changing permissions of âCMSTEST.EXECâ: Function not
> > implemented
> > >>>> chmod: changing permissions of âMICHAELW.PDFâ: Function not
> > implemented
> > >>>> chmod: changing permissions of âPROFILE.EXECâ: Function not
> > implemented
> > >>>> chmod: changing permissions of âPROFILE2.EXECâ: Function not
> > implemented
> > >>>>
> > >>>> I also tried
> > >>>>
> > >>>> sles12:/srv/www/htdocs/test # cmsfs-fuse -o mask=644 *
> > >>>> cmsfs-fuse: Error CMS1 label not found!
> > >>>>
> > >>>> Any suggestions? I formatted the DISK in z/VM with format 191 a and
> > >>>> created a label. I am a bit stumped.
> > >>>>
> > >>>>
> > >>>> Sent from my iPhone
> > >>>>
> --
> > >>>> For LINUX-390 subscribe / signoff / archive access instructions,
> > >>>> send email to lists...@vm.marist.edu with the message: INFO
> LINUX-390
> > >> or
> > >>>> visit
> > >>>> http://www.marist.edu/htbin/wlvindex?LINUX-390
> > >>>>
> --
> > >>>> For more information on Linux on System z, visit
> > >>>> http://wiki.linuxvm.org/
> > &

Re: Cmsfs

[Michael MacIsaac]

Mike,

Either add a specific directory section for /srv/www/htdocs/test or it will
inherit from /srv/www/htdocs which is defined in
/etc/apache2/default-server.conf.

Who knows, maybe it's working now. What do you get when you specify a URL
such as http://your.server/test/MICHAELW.PDF

If you get an error, look at tail of the updated log files in
/var/log/apache2

HTH

-Mike

On Thu, Mar 3, 2016 at 8:21 AM, Michael Weiner <mwei...@infinite-blue.com>
wrote:

> No I don't believe that I do.
>
> Do I need to add?
>
> Sent from my iPhone
>
> > On Mar 3, 2016, at 8:13 AM, Michael MacIsaac <mike99...@gmail.com>
> wrote:
> >
> > Mike,
> >
> >> I want to set these permissions so I can view the files using Apache.
> > Do you have a Directory section in the Apache conf file for
> > /srv/www/htdocs/test?
> >
> > If so does it have:
> >Order allow,deny
> >Allow from all
> >
> >-Mike M
> >
> > On Thu, Mar 3, 2016 at 8:02 AM, Michael Weiner <
> mwei...@infinite-blue.com>
> > wrote:
> >
> >> I tried setting the permissions of the folder.
> >>
> >> I want to set these permissions so I can view the files using Apache.
> >>
> >> Sent from my iPhone
> >>
> >>>> On Mar 3, 2016, at 7:47 AM, Michael MacIsaac <mike99...@gmail.com>
> >>> wrote:
> >>>
> >>> Mike,
> >>>
> >>> Why do you want to set permissions on CMS files?
> >>>
> >>>> Any suggestions?
> >>> Can you set the permissions of the mount point, and they will pertain
> to
> >>> the mounted CMS files?
> >>>
> >>>   -Mike M
> >>>
> >>> On Thu, Mar 3, 2016 at 7:32 AM, Michael Weiner <
> >> mwei...@infinite-blue.com>
> >>> wrote:
> >>>
> >>>> I think I posted my question in the wrong group.
> >>>>
> >>>> Scott, thanks for your help.
> >>>>
> >>>> According to the documentation the cmsfuse unmask command should be
> able
> >>>> to change the permissions. Two things I tried.
> >>>>
> >>>> sles12:/srv/www/htdocs/test # chmod 644 *
> >>>> chmod: changing permissions of âCMSTEST.EXECâ: Function not
> implemented
> >>>> chmod: changing permissions of âMICHAELW.PDFâ: Function not
> implemented
> >>>> chmod: changing permissions of âPROFILE.EXECâ: Function not
> implemented
> >>>> chmod: changing permissions of âPROFILE2.EXECâ: Function not
> implemented
> >>>>
> >>>> I also tried
> >>>>
> >>>> sles12:/srv/www/htdocs/test # cmsfs-fuse -o mask=644 *
> >>>> cmsfs-fuse: Error CMS1 label not found!
> >>>>
> >>>> Any suggestions? I formatted the DISK in z/VM with format 191 a and
> >>>> created a label. I am a bit stumped.
> >>>>
> >>>>
> >>>> Sent from my iPhone
> >>>> --
> >>>> For LINUX-390 subscribe / signoff / archive access instructions,
> >>>> send email to lists...@vm.marist.edu with the message: INFO LINUX-390
> >> or
> >>>> visit
> >>>> http://www.marist.edu/htbin/wlvindex?LINUX-390
> >>>> --
> >>>> For more information on Linux on System z, visit
> >>>> http://wiki.linuxvm.org/
> >>>
> >>> --
> >>> For LINUX-390 subscribe / signoff / archive access instructions,
> >>> send email to lists...@vm.marist.edu with the message: INFO LINUX-390
> >> or visit
> >>> http://www.marist.edu/htbin/wlvindex?LINUX-390
> >>> --
> >>> For more information on Linux on System z, visit
> >>> http://wiki.linuxvm.org/
> >>
> >> --
> >> For LINUX-390 subscribe / signoff / archive access instructions,
> >> send email to lists...@vm.marist.edu with the message: INFO LINUX-390
> or
> >> visit
> >> http://www.marist.edu/htbin/wlvindex?LINUX-390
> >> --
> >> For more information on Linux on System z, visit
&g

Re: Cmsfs

[Michael Weiner]

No I don't believe that I do. 

Do I need to add?

Sent from my iPhone

> On Mar 3, 2016, at 8:13 AM, Michael MacIsaac <mike99...@gmail.com> wrote:
> 
> Mike,
> 
>> I want to set these permissions so I can view the files using Apache.
> Do you have a Directory section in the Apache conf file for
> /srv/www/htdocs/test?
> 
> If so does it have:
>Order allow,deny
>Allow from all
> 
>-Mike M
> 
> On Thu, Mar 3, 2016 at 8:02 AM, Michael Weiner <mwei...@infinite-blue.com>
> wrote:
> 
>> I tried setting the permissions of the folder.
>> 
>> I want to set these permissions so I can view the files using Apache.
>> 
>> Sent from my iPhone
>> 
>>>> On Mar 3, 2016, at 7:47 AM, Michael MacIsaac <mike99...@gmail.com>
>>> wrote:
>>> 
>>> Mike,
>>> 
>>> Why do you want to set permissions on CMS files?
>>> 
>>>> Any suggestions?
>>> Can you set the permissions of the mount point, and they will pertain to
>>> the mounted CMS files?
>>> 
>>>   -Mike M
>>> 
>>> On Thu, Mar 3, 2016 at 7:32 AM, Michael Weiner <
>> mwei...@infinite-blue.com>
>>> wrote:
>>> 
>>>> I think I posted my question in the wrong group.
>>>> 
>>>> Scott, thanks for your help.
>>>> 
>>>> According to the documentation the cmsfuse unmask command should be able
>>>> to change the permissions. Two things I tried.
>>>> 
>>>> sles12:/srv/www/htdocs/test # chmod 644 *
>>>> chmod: changing permissions of âCMSTEST.EXECâ: Function not implemented
>>>> chmod: changing permissions of âMICHAELW.PDFâ: Function not implemented
>>>> chmod: changing permissions of âPROFILE.EXECâ: Function not implemented
>>>> chmod: changing permissions of âPROFILE2.EXECâ: Function not implemented
>>>> 
>>>> I also tried
>>>> 
>>>> sles12:/srv/www/htdocs/test # cmsfs-fuse -o mask=644 *
>>>> cmsfs-fuse: Error CMS1 label not found!
>>>> 
>>>> Any suggestions? I formatted the DISK in z/VM with format 191 a and
>>>> created a label. I am a bit stumped.
>>>> 
>>>> 
>>>> Sent from my iPhone
>>>> --
>>>> For LINUX-390 subscribe / signoff / archive access instructions,
>>>> send email to lists...@vm.marist.edu with the message: INFO LINUX-390
>> or
>>>> visit
>>>> http://www.marist.edu/htbin/wlvindex?LINUX-390
>>>> --
>>>> For more information on Linux on System z, visit
>>>> http://wiki.linuxvm.org/
>>> 
>>> --
>>> For LINUX-390 subscribe / signoff / archive access instructions,
>>> send email to lists...@vm.marist.edu with the message: INFO LINUX-390
>> or visit
>>> http://www.marist.edu/htbin/wlvindex?LINUX-390
>>> --
>>> For more information on Linux on System z, visit
>>> http://wiki.linuxvm.org/
>> 
>> --
>> For LINUX-390 subscribe / signoff / archive access instructions,
>> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
>> visit
>> http://www.marist.edu/htbin/wlvindex?LINUX-390
>> --
>> For more information on Linux on System z, visit
>> http://wiki.linuxvm.org/
> 
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Cmsfs

[Michael MacIsaac]

Mike,

> I want to set these permissions so I can view the files using Apache.
Do you have a Directory section in the Apache conf file for
/srv/www/htdocs/test?

If so does it have:
Order allow,deny
Allow from all

-Mike M

On Thu, Mar 3, 2016 at 8:02 AM, Michael Weiner <mwei...@infinite-blue.com>
wrote:

> I tried setting the permissions of the folder.
>
> I want to set these permissions so I can view the files using Apache.
>
> Sent from my iPhone
>
> > On Mar 3, 2016, at 7:47 AM, Michael MacIsaac <mike99...@gmail.com>
> wrote:
> >
> > Mike,
> >
> > Why do you want to set permissions on CMS files?
> >
> >> Any suggestions?
> > Can you set the permissions of the mount point, and they will pertain to
> > the mounted CMS files?
> >
> >-Mike M
> >
> > On Thu, Mar 3, 2016 at 7:32 AM, Michael Weiner <
> mwei...@infinite-blue.com>
> > wrote:
> >
> >> I think I posted my question in the wrong group.
> >>
> >> Scott, thanks for your help.
> >>
> >> According to the documentation the cmsfuse unmask command should be able
> >> to change the permissions. Two things I tried.
> >>
> >> sles12:/srv/www/htdocs/test # chmod 644 *
> >> chmod: changing permissions of âCMSTEST.EXECâ: Function not implemented
> >> chmod: changing permissions of âMICHAELW.PDFâ: Function not implemented
> >> chmod: changing permissions of âPROFILE.EXECâ: Function not implemented
> >> chmod: changing permissions of âPROFILE2.EXECâ: Function not implemented
> >>
> >> I also tried
> >>
> >> sles12:/srv/www/htdocs/test # cmsfs-fuse -o mask=644 *
> >> cmsfs-fuse: Error CMS1 label not found!
> >>
> >> Any suggestions? I formatted the DISK in z/VM with format 191 a and
> >> created a label. I am a bit stumped.
> >>
> >>
> >> Sent from my iPhone
> >> --
> >> For LINUX-390 subscribe / signoff / archive access instructions,
> >> send email to lists...@vm.marist.edu with the message: INFO LINUX-390
> or
> >> visit
> >> http://www.marist.edu/htbin/wlvindex?LINUX-390
> >> --
> >> For more information on Linux on System z, visit
> >> http://wiki.linuxvm.org/
> >
> > --
> > For LINUX-390 subscribe / signoff / archive access instructions,
> > send email to lists...@vm.marist.edu with the message: INFO LINUX-390
> or visit
> > http://www.marist.edu/htbin/wlvindex?LINUX-390
> > --
> > For more information on Linux on System z, visit
> > http://wiki.linuxvm.org/
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Cmsfs

[Michael Weiner]

I tried setting the permissions of the folder. 

I want to set these permissions so I can view the files using Apache. 

Sent from my iPhone

> On Mar 3, 2016, at 7:47 AM, Michael MacIsaac <mike99...@gmail.com> wrote:
> 
> Mike,
> 
> Why do you want to set permissions on CMS files?
> 
>> Any suggestions?
> Can you set the permissions of the mount point, and they will pertain to
> the mounted CMS files?
> 
>-Mike M
> 
> On Thu, Mar 3, 2016 at 7:32 AM, Michael Weiner <mwei...@infinite-blue.com>
> wrote:
> 
>> I think I posted my question in the wrong group.
>> 
>> Scott, thanks for your help.
>> 
>> According to the documentation the cmsfuse unmask command should be able
>> to change the permissions. Two things I tried.
>> 
>> sles12:/srv/www/htdocs/test # chmod 644 *
>> chmod: changing permissions of âCMSTEST.EXECâ: Function not implemented
>> chmod: changing permissions of âMICHAELW.PDFâ: Function not implemented
>> chmod: changing permissions of âPROFILE.EXECâ: Function not implemented
>> chmod: changing permissions of âPROFILE2.EXECâ: Function not implemented
>> 
>> I also tried
>> 
>> sles12:/srv/www/htdocs/test # cmsfs-fuse -o mask=644 *
>> cmsfs-fuse: Error CMS1 label not found!
>> 
>> Any suggestions? I formatted the DISK in z/VM with format 191 a and
>> created a label. I am a bit stumped.
>> 
>> 
>> Sent from my iPhone
>> --
>> For LINUX-390 subscribe / signoff / archive access instructions,
>> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
>> visit
>> http://www.marist.edu/htbin/wlvindex?LINUX-390
>> --
>> For more information on Linux on System z, visit
>> http://wiki.linuxvm.org/
> 
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Cmsfs

[Michael MacIsaac]

Mike,

Why do you want to set permissions on CMS files?

> Any suggestions?
Can you set the permissions of the mount point, and they will pertain to
the mounted CMS files?

-Mike M

On Thu, Mar 3, 2016 at 7:32 AM, Michael Weiner <mwei...@infinite-blue.com>
wrote:

> I think I posted my question in the wrong group.
>
> Scott, thanks for your help.
>
> According to the documentation the cmsfuse unmask command should be able
> to change the permissions. Two things I tried.
>
> sles12:/srv/www/htdocs/test # chmod 644 *
> chmod: changing permissions of âCMSTEST.EXECâ: Function not implemented
> chmod: changing permissions of âMICHAELW.PDFâ: Function not implemented
> chmod: changing permissions of âPROFILE.EXECâ: Function not implemented
> chmod: changing permissions of âPROFILE2.EXECâ: Function not implemented
>
> I also tried
>
> sles12:/srv/www/htdocs/test # cmsfs-fuse -o mask=644 *
> cmsfs-fuse: Error CMS1 label not found!
>
> Any suggestions? I formatted the DISK in z/VM with format 191 a and
> created a label. I am a bit stumped.
>
>
> Sent from my iPhone
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
> visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Cmsfs

[Michael Weiner]

I think I posted my question in the wrong group. 

Scott, thanks for your help. 

According to the documentation the cmsfuse unmask command should be able to 
change the permissions. Two things I tried. 

sles12:/srv/www/htdocs/test # chmod 644 *
chmod: changing permissions of âCMSTEST.EXECâ: Function not implemented
chmod: changing permissions of âMICHAELW.PDFâ: Function not implemented
chmod: changing permissions of âPROFILE.EXECâ: Function not implemented
chmod: changing permissions of âPROFILE2.EXECâ: Function not implemented

I also tried

sles12:/srv/www/htdocs/test # cmsfs-fuse -o mask=644 *
cmsfs-fuse: Error CMS1 label not found!

Any suggestions? I formatted the DISK in z/VM with format 191 a and created a 
label. I am a bit stumped.


Sent from my iPhone
--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Bug in cmsfs-fuse?

[Michael Holzheu]

Hello Mike,

Thanks for your detailed report.

The CMS file system stores time stamps in local time. The cmsfs-fuse
file system has to report time in UTC to the Linux VFS layer. Therefore
local time has to be converted to UTC for reading and UTC to local time
for writing. IMO this is currently done incorrectly.

In case you are interested, we can provide the corresponding s390-tools patch.

Regards,
Michael

On Fri, 4 Sep 2015 11:18:19 -0400
Michael MacIsaac <mike99...@gmail.com> wrote:

> Cross-posted to IBMVM and linux-390
>
> Is this a bug in cmsfs-fuse?  It looks like the timestamp of a file created
> on Linux is an hour ahead - note 12:03:12 vs. 11:03:12 below:
>
> On VM:
> -) Link and access a CMS filesystem (THEVIRTM 1191) as F from VM
> -) Create a file with a timestamp:
> ==> pipe cp query time |> thetime fromvm f
> Ready; T=0.01/0.01 10:59:28
> ==> rel f (det
> DASD 1191 DETACHED
>
> On a Linux running on the VM with the CMS disk:
> # vmcp link thevirtm 1191 1191 mr
> # chccwdev -e 1191
> Setting device 0.0.1191 online
> Done
> # lsdasd | grep 1191
> 0.0.1191   active  dasde 94:16   ECKD  4096   70MB  18000
> # mkdir /srv/CMSdisk
> # cmsfs-fuse -a /dev/dasde /srv/CMSdisk
> # date > /srv/CMSdisk/thetime.onlinux
> # fusermount -u /srv/CMSdisk/
> # chccwdev -d 1191
> Setting device 0.0.1191 offline
> Done
> # vmcp DETACH 1191
> DASD 1191 DETACHED
>
> Back on VM:
> ==> vmlink thevirtm 1191
> ==> filel * * z
>   THETIME  ONLINUX  Z1 V 28  1  1  9/04/15
> 12:03:12
>   THETIME  FROMVM   Z1 V 54  2  1  9/04/15
> 10:59:28
> ==> type THETIME  FROMVM   Z
>
> TIME IS 10:59:28 EDT FRIDAY 09/04/15
> CONNECT= 00:06:13 VIRTCPU= 000:00.00 TOTCPU= 000:00.01
>
> ==> type THETIME  ONLINUX  Z
>
> Fri Sep  4 11:03:12 EDT 2015
>
> -Mike M
>
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/
>

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Bug in cmsfs-fuse?

[Michael MacIsaac]

Cross-posted to IBMVM and linux-390

Is this a bug in cmsfs-fuse?  It looks like the timestamp of a file created
on Linux is an hour ahead - note 12:03:12 vs. 11:03:12 below:

On VM:
-) Link and access a CMS filesystem (THEVIRTM 1191) as F from VM
-) Create a file with a timestamp:
==> pipe cp query time |> thetime fromvm f
Ready; T=0.01/0.01 10:59:28
==> rel f (det
DASD 1191 DETACHED

On a Linux running on the VM with the CMS disk:
# vmcp link thevirtm 1191 1191 mr
# chccwdev -e 1191
Setting device 0.0.1191 online
Done
# lsdasd | grep 1191
0.0.1191   active  dasde 94:16   ECKD  4096   70MB  18000
# mkdir /srv/CMSdisk
# cmsfs-fuse -a /dev/dasde /srv/CMSdisk
# date > /srv/CMSdisk/thetime.onlinux
# fusermount -u /srv/CMSdisk/
# chccwdev -d 1191
Setting device 0.0.1191 offline
Done
# vmcp DETACH 1191
DASD 1191 DETACHED

Back on VM:
==> vmlink thevirtm 1191
==> filel * * z
  THETIME  ONLINUX  Z1 V 28  1  1  9/04/15
12:03:12
  THETIME  FROMVM   Z1 V 54  2  1  9/04/15
10:59:28
==> type THETIME  FROMVM   Z

TIME IS 10:59:28 EDT FRIDAY 09/04/15
CONNECT= 00:06:13 VIRTCPU= 000:00.00 TOTCPU= 000:00.01

==> type THETIME  ONLINUX  Z

Fri Sep  4 11:03:12 EDT 2015

-Mike M

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

cmsfs woes

[Michael MacIsaac]

Hello list,

Has anyone heard of cmsfscat or cmsfslst corrupting a CMS disk?  A few of
us were sharing a common 191 disk R/W occasionally on CMS, but R/O on
Linux (by definition).

I ran a script that uses cmsfs and got this:

cmsfs_map_ADT(): directory RECFM 'V' not 'F'
cmsfs_vopen(): cmsfs_map_ADT() returned -1
*** glibc detected *** cmsfscat: double free or corruption (!prev):
0x80009180 ***
=== Backtrace: =
/lib64/libc.so.6[0x20ac46e]
/lib64/libc.so.6(__libc_free+0x82)[0x20ad9fe]
cmsfscat[0x8000185c]
cmsfscat[0x8000198a]
cmsfscat[0x8e8a]
/lib64/libc.so.6(__libc_start_main+0x100)[0x2051568]
cmsfscat(__libc_start_main+0x5e)[0x8c1a]
=== Memory map: 
8000-80005000 r-xp  5e:05 119256 /usr/bin/cmsfscat
80005000-80006000 rw-p 4000 5e:05 119256 /usr/bin/cmsfscat
80006000-8002a000 rwxp 80006000 00:00 0 [heap]
200-201e000 r-xp  5e:05 306443 /lib64/ld-2.4.so
...

Now the common CMS disk is giving a device error. Not sure if this is
the cause or the symptom. Any help will be appreciated.

Mike MacIsaac mikemac at-sign us.ibm.com

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs woes

[David Kreuter]

sorry for your troubles. Can't help but think that somehow multi-write
was involved here. What is meant by 
sharing a common 191 disk r/w - simultaneously r/w'ing from two CMS
machines? ouch
David Kreuter


 Original Message 
Subject: cmsfs woes
From: Michael MacIsaac mike...@us.ibm.com
Date: Mon, September 24, 2012 11:59 am
To: LINUX-390@VM.MARIST.EDU

Hello list,

Has anyone heard of cmsfscat or cmsfslst corrupting a CMS disk? A few of
us were sharing a common 191 disk R/W occasionally on CMS, but R/O on
Linux (by definition).

I ran a script that uses cmsfs and got this:

cmsfs_map_ADT(): directory RECFM 'V' not 'F'
cmsfs_vopen(): cmsfs_map_ADT() returned -1
*** glibc detected *** cmsfscat: double free or corruption (!prev):
0x80009180 ***
=== Backtrace: =
/lib64/libc.so.6[0x20ac46e]
/lib64/libc.so.6(__libc_free+0x82)[0x20ad9fe]
cmsfscat[0x8000185c]
cmsfscat[0x8000198a]
cmsfscat[0x8e8a]
/lib64/libc.so.6(__libc_start_main+0x100)[0x2051568]
cmsfscat(__libc_start_main+0x5e)[0x8c1a]
=== Memory map: 
8000-80005000 r-xp  5e:05 119256 /usr/bin/cmsfscat
80005000-80006000 rw-p 4000 5e:05 119256 /usr/bin/cmsfscat
80006000-8002a000 rwxp 80006000 00:00 0 [heap]
200-201e000 r-xp  5e:05 306443 /lib64/ld-2.4.so
...

Now the common CMS disk is giving a device error. Not sure if this is
the cause or the symptom. Any help will be appreciated.

Mike MacIsaac mikemac at-sign us.ibm.com

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs woes

[Michael MacIsaac]

David,

 sharing a common 191 disk r/w - simultaneously r/w'ing from two CMS
 machines? ouch
Poor choice of words.  Link mode was MR, so only one gets to write at a
time.

I have since learned that DDR to this disk was somehow involved, and that
is probably the smoking gun.  It does sense that a R/O CMS file system
tool cannot corrupt a disk.

Still, cmsfs could have returned something like unable to read disk,
rather than what it did.  Serious boundary test case though ...

Mike MacIsaac mikemac at-sign us.ibm.com

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Jan Glauber]

On Thu, 2011-12-15 at 12:03 -0600, David Boyes wrote:
 Look at how DIAG 250 works in the CP Programming Services manual to
 get an idea of a way to approach this with minimum memory usage.

That sounds interesting. I'll add that to my things-to-improve list ;-

Jan

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Jan Glauber]

On Wed, 2011-12-14 at 13:40 +0100, Rob van der Heij wrote:
 On Wed, Dec 14, 2011 at 1:20 PM, Jan Glauber j...@linux.vnet.ibm.com wrote:

  Since there seems to be collective disapproval of the requirement to
  touch the memory settings for large disks I'm looking into changing
  that...

 You don't get a random sample of the impacted audience  ;-)   While
 the ideal world has no limitations, no rain when you walk the dog and
 good performance every day, you can't always have it. You probably do
 need to document the limitation and have a useful error message. But
 if it hurts, then don't do it.

Thanks Rob ;- I've indeed documented this limitation for our new Device
Drivers book, p. 485:
http://public.dhe.ibm.com/software/dw/linux390/docu/l3n1dd13.pdf

But I guess everybody including myself prefers things that work
out-of-the box...

Jan


 I think we can agree that CMS mini disks of 20 GB is beyond common
 usage and intended purpose of cmsfs. In fact, CMS will show its
 limitations as well when you try to do something with it. When looking
 at where to put your energy, I have a dozen other things that seem far
 more significant to make the platform work...

 Rob (from the peanut gallery)

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Jan Glauber]

On Wed, 2011-12-14 at 11:39 -0700, Mark Post wrote:
  On 12/14/2011 at 07:40 AM, Rob van der Heij rvdh...@gmail.com wrote:
  I think we can agree that CMS mini disks of 20 GB is beyond common
  usage and intended purpose of cmsfs.

 Is 2.3GB too much to ask?  That was my situation, and I found it rather 
 irritating even then.

Well, kind of depends on the distributions settings:

h4245006:~ # cat /etc/SuSE-release
SUSE Linux Enterprise Server 11 (s390x)
VERSION = 11
PATCHLEVEL = 1

h4245006:~ # ulimit -v
816320

I can't image cmsfs-fuse is the only program that will hit that limit
by creating a memory maping bigger than ~800MB.

Jan


 Mark Post

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Jan Glauber]

On Wed, 2011-12-14 at 11:43 -0700, Mark Post wrote:
  On 12/14/2011 at 07:20 AM, Jan Glauber j...@linux.vnet.ibm.com wrote:

  I can easily replace the mmap-memcpy with pread/pwrite. Unfortunately I
  see huge performance drops if I do so. Currently I'm looking why the
  system call variant costs so much more than mmap.

 Define huge performance drops.  I don't know that most people are going to 
 be reading large files from a CMS minidisk.  They're more likely to have 
 large disks with lots of reasonably sized files.  I suspect that some people 
 are wanting to read very large files, but I don't know how many people, or 
 how large the files are.

I've seen read performance go down by a factor of ten by using pread vs.
memcpy. But this measurement seems not worth the bytes I'm currently
wasting. Today everything is fast (enough) again, regardless of pread or
memcpy. I suspect there was something fishy with the disk I'm using.

While I would not like to introduce something that costs factor of 10
I think CMS EDF is not a high-performance file system and neither is
cmsfs-fuse for various reasons.

 Would it be possible to use pread for the FST, and then mmap-memcpy the 
 actual file being read?  I would find it far more understandable to need more 
 memory to read a very large file than I would to just mount the file system.

That would not be possible since files (and also FST's and the
allocation map) could consist of (nearly) any block on the disk so
they are not contiguous. And mmap'ing single blocks would not be worth
the effort.

Jan


 Mark Post

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Rob van der Heij]

On Thu, Dec 15, 2011 at 2:05 PM, Jan Glauber j...@linux.vnet.ibm.com wrote:

 I can't image cmsfs-fuse is the only program that will hit that limit
 by creating a memory maping bigger than ~800MB.

You could not even run FF8 with that, I fear...  Maybe that's the
reason the JVM unzips all the files into private memory before use
;-)

I think the really big stuff does not rely on the default of the
distribution. Things like Oracle come with their own setup
instructions for example to set ulimit -v to allow for 4G mapped, I
guess you need to raise that even further when you grow the SGA beyond
that.

Rob

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Jan Glauber]

On Wed, 2011-12-14 at 16:27 -0500, Richard Troth wrote:
 I suggest two things:
 First, draw a line (this would be a somewhat arbitrary number).  If
 the disk is less than (for example) 256M (roughly 350 cyls of 3390),
 then continue to use mmap/memcpy.  If larger, then switch to
 pread/pwrite.  Second, provide a mount option so that the user can
 always specify which model to they want.  (With the option to be
 explicit, there is little reason any of us can complain about where
 you draw that line.)

I like that idea of using both methods, but I don't like the artificial
limit.

What I would like to do is the following:

1. mmap the disk (as before)
2. check for the -ENOMEM case
   2a. mmap was successful - go on use memcpy in the read/write
functions
   2b. mmap failed - use pread/pwrite in the read/write functions

That way we can avoid any memory setting changes since pread/pwrite will
also work with very little memory. If the sysadmin sets ulimit -v
cmsfs-fuse will also work. And in case there is enough memory it can be
used to cache the disk operations.

Is anybody ok with that approach or do you still see the need for an
additional switch???

Jan

 -- R;   
 Rick Troth
 Velocity Software
 http://www.velocitysoftware.com/





 On Wed, Dec 14, 2011 at 07:20, Jan Glauber j...@linux.vnet.ibm.com wrote:
  On Fri, 2011-12-02 at 14:05 -0600, David Boyes wrote:
   It's just mmap'ing the whole disk into the process's address space for 
   the
   programmers sake.
 
  BAD idea (in the sense of 'broken as designed'). You're taxing the virtual 
  memory system in a shared-resource environment, which hurts the entire 
  environment for a little convenience in programming.
 
   If that turns out to be a problem we could theoretically go back to
   pread/pwrite. But I'm not sure how many users have such large CMS disks?
 
  Please do. Doesn't matter if it's an edge case, it shouldn't do this.
 
  Since there seems to be collective disapproval of the requirement to
  touch the memory settings for large disks I'm looking into changing
  that...
 
  I can easily replace the mmap-memcpy with pread/pwrite. Unfortunately I
  see huge performance drops if I do so. Currently I'm looking why the
  system call variant costs so much more than mmap.
 
  Jan
 
  --
  For LINUX-390 subscribe / signoff / archive access instructions,
  send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or 
  visit
  http://www.marist.edu/htbin/wlvindex?LINUX-390
  --
  For more information on Linux on System z, visit
  http://wiki.linuxvm.org/
 
 
  --
  For LINUX-390 subscribe / signoff / archive access instructions,
  send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or 
  visit
  http://www.marist.edu/htbin/wlvindex?LINUX-390
  --
  For more information on Linux on System z, visit
  http://wiki.linuxvm.org/

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Jan Glauber]

On Wed, 2011-12-14 at 12:29 -0600, David Boyes wrote:
  I can easily replace the mmap-memcpy with pread/pwrite. Unfortunately I
  see huge performance drops if I do so. Currently I'm looking why the system
  call variant costs so much more than mmap.

 An alternative approach would be to mmap *only* the minidisk label and FST on 
 access of the minidisk, and then mmap individual files as you need them. I'm 
 not objecting to the use of mmap, just the gratuitous use of mmap on the 
 whole disk rather than what you actually *need* at that moment.

No, that wont work, files and the FST (also the allocation map) can be
spread across any blocks of the disk. And mmap'ing single blocks
together is complex and not worth the effort.

Please see the other thread for my proposal...

Jan

 It's like the old Antonin Chekov quote about what's necessary in a play: if 
 there's a pistol on stage, you need to have killed someone by act II, scene 
 1. Otherwise, take it out -- it didn't need to be there.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[David Boyes]

 While I would not like to introduce something that costs factor of 10 I think
 CMS EDF is not a high-performance file system and neither is cmsfs-fuse for
 various reasons.

Natively EDF is pretty good as single user OSes go. CMS VSAM... well, there's a 
tale for others to tell. 
 
 That would not be possible since files (and also FST's and the allocation map)
 could consist of (nearly) any block on the disk so they are not contiguous.
 And mmap'ing single blocks would not be worth the effort.

I guess I don't understand this comment. Mapping the label and running the FST 
to determine what blocks to map seems worthwhile, since you could use common 
code to mmap the blocks of files you want to read into a contiguous memory area 
for efficient access. I don't see why that couldn't be a contiguous struct in 
Linux space regardless of where the blocks actually are on the disk.  For R/O 
access, you generally don't care about updates to the FST until the next access 
of the minidisk (standard CMS semantics), and for R/W, you have transactional 
control of the blocks updated, and you need to update the FST anyway, so you 
want to be able to run that chain efficiently to find the right places to 
update.

Look at how DIAG 250 works in the CP Programming Services manual to get an idea 
of a way to approach this with minimum memory usage. The way DIAG 250 handles 
it (essentially assemble a list of block operations (read AND write) and a 
memory buffer to populate, then execute the I/O to fill the buffer, then hand 
it back to the application logic to parse the memory buffer) is extremely 
efficient on memory and CPU, works nicely on LPAR and VM, and would work 
equally well for the minidisk access routines and file access (both read-only 
and read-write).  The FST gives you sufficient information to determine the 
size of the buffer needed to mmap the file you're accessing (in Mark's example 
case of a large disk with lots of small files on it -- probably the most common 
use case IMHO -- you probably want one or maybe two files off that disk). Note: 
I'm talking about only the I/O list logic in the DIAG 250 description -- I am 
explicitly *not* asking you to do a clone of DIAG 250.. 8-)

That gives you a use-only-what-you need approach and then the size of the 
minidisk doesn't ever matter. 

-- db
 

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Mark Post]

 On 12/15/2011 at 08:33 AM, Jan Glauber j...@linux.vnet.ibm.com wrote: 
 1. mmap the disk (as before)
 2. check for the -ENOMEM case
2a. mmap was successful - go on use memcpy in the read/write
 functions
2b. mmap failed - use pread/pwrite in the read/write functions
 
 That way we can avoid any memory setting changes since pread/pwrite will
 also work with very little memory. If the sysadmin sets ulimit -v
 cmsfs-fuse will also work. And in case there is enough memory it can be
 used to cache the disk operations.
 
 Is anybody ok with that approach or do you still see the need for an
 additional switch???

I'm willing to say it's good for a first whack at the problem.  Further testing 
may show that something more (or else) is needed.  Either way, I'll report back 
what I find.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[David Boyes]

 -Original Message-
 From: Linux on 390 Port [mailto:LINUX-390@VM.MARIST.EDU] On Behalf Of
 Rob van der Heij
 As Jan points out the FST is fragmented.

Agreed. However, each piece contains pointers to the next piece you need, and 
you need that information anyway, so following the breadcrumbs is not an 
operational loss as it happens in two scenarios: at first access, and after a 
update to the FST in R/W mode.

 The purpose of mmap() is that you
 map all blocks in virtual mem 

The purpose of mmap() is to map *A* specified object (disk, shared memory 
block, etc) of a specified size starting at a specified offset from the start 
of the object to *A* memory segment of equal size in the process' address 
space. It does not have to map ALL blocks of a disk to access some of the data 
on it. 

POSIX (IEEE Std 1003.1) definition of mmap():

The mmap() function shall establish a mapping between a process' address space 
and a file, shared memory object, or typed memory object. The format of the 
call is as follows:

pa=mmap(addr, len, prot, flags, fildes, off);

The mmap() function shall establish a mapping between the address space of the 
process at an address pa for len bytes to the memory object represented by the 
file descriptor fildes at offset off for len bytes. The value of pa is an 
implementation-defined function of the parameter addr and the values of flags, 
further described below. A successful mmap() call shall return pa as its 
result. The address range starting at pa and continuing for len bytes shall be 
legitimate for the possible (not necessarily current) address space of the 
process. The range of bytes starting at off and continuing for len bytes shall 
be legitimate for the possible (not necessarily current) offsets in the file, 
shared memory object, or typed memory object represented by fildes.

 and then simply access the blocks in memory.
 Linux does the I/O under the covers.

I follow the concept, and see the advantages of using mmap to do the I/O under 
the covers. At this point, we're optimizing to minimize the amount of data we 
need, and thus the impact on other stuff that uses memory in the same virtual 
machine (and WSS of same).

 Since your blocks can be anywhere on
 disk, you map the entire thing. 

Here's where we diverge. 

There are two issues here: 

1) accessing the minidisk and representing its contents to Linux at a point in 
time
2) accessing the content of the minidisk

Mmap()ing the whole disk is a convenient solution to both problems, HOWEVER:

To access a minidisk and represent it to Linux, you do NOT need every block on 
the disk to be represented in a structure, you need the label data and the FST 
data (which, btw you need to read first ANYWAY to mmap the whole disk as you 
need to know the logical number of blocks to set up the mmap!). 

To use the files on the minidisk, you need the blocks CONTAINED in the file, 
not the entire disk. You get that from the FST and you mmap() those blocks.

Quote (again from IEEE Std 1003.1): Use of mmap() may reduce the amount of 
memory available to other memory allocation functions. 

This is what triggered the discussion. In no case do you ever need the entire 
set of blocks on the disk at the same time, unless they are contained in a 
single large file, which our use case (big disk with lots of small to 
medium-size files) makes unlikely, if not explicitly impossible by definition 
of the problem. 

To map just record 3-5 is no gain if you need
 to point still at the rest of the blocks.

See above. It is a gain at access time (you don't need ALL the blocks, you need 
the ones to identify the volume, create a view of the volume contents, and 
where the interesting content is, or at least starts).  For R/O you need to 
build an in memory copy exactly once (on first access to the minidisk, then you 
can use it forever until the next access). For R/W you need a live copy in 
memory of the entire FST, which you need to build and maintain, regardless of 
activity or access method. The in-memory copy does not have to be discontiguous 
-- in fact, you *want* it to be contiguous so you can use simple indexing of a 
structure pattern over the FST entries for performance. 

You don't need the other data AT ALL until you actually access a file in some 
way, and then you need only the blocks that comprise the file you want. 

 DIAG250 is a block driver, just like Linux can do. Extra work is to allocate
 memory to hold blocks while you work on them, make sure to flush the
 updated pages, etc.

I suggested looking at DIAG 250 for ideas on how to approach the problem. I 
explicitly said that I do not want a duplicate of DIAG 250. 

Yes, it's going to be a little more work on the housekeeping tasks if you want 
R/W access, but you'd have to do substantially the same housekeeping with the 
full-disk map  approach. It's mostly buffer management issues, though; the 
actual update to the data on disk can still be done with memcpy 

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Jan Glauber]

On Fri, 2011-12-02 at 18:10 -0200, Rogério Soares wrote:
 under cms without problem apparently, i made a loop for try copy, the most
 files i can´t copy ... :-/

Rogerio, we've released a new version of s390-tools that contains
several cmsfs-fuse bugfixes. I guess you've been running your own
s390-tools version since cmsfs-fuse is not yet officially released for
SLES11. Do you mind re-testing with the newer cmsfs-fuse version?

You can download it here:
http://www.ibm.com/developerworks/linux/linux390/s390-tools-1.16.0.html

thanks, Jan

 
 On Fri, Dec 2, 2011 at 6:00 PM, Jan Glauber j...@linux.vnet.ibm.com wrote:
 
  On Fri, 2011-12-02 at 16:36 -0200, Rogério Soares wrote:
   Jan,
  
   i had mount using:
  
   cmsfs-fuse /dev/dasdc /linmon/ -o allow_root
  
   capp101:/linmon # cmsfs-fuse /dev/dasdc /linmon/ -o allow_root
  
   when, i tried copy files to another location, i got:
  
   capp101:/linmon # cp Z196PROD.MD000675 /root/
  
   cp: cannot open `Z196PROD.MD000675' for reading: Input/output error
  
   is it supposed to be ok?
 
  Well, seriously not. This looks like a bug or a broken file.
 
  Can you access the file under CMS? Does this happen for every file on
  the disk or only for this file?
 
  Jan
 
  
  
   On Fri, Dec 2, 2011 at 4:19 PM, Mark Post mp...@novell.com wrote:
  
 On 12/2/2011 at 12:26 PM, Jan Glauber j...@linux.vnet.ibm.com
  wrote:
 Try to increase the limit, it should be bigger than the size of the
  disk
 you're trying to mount (or unlimited).

 If you don't have enough free memory in your guest you may also need
  to
 set  /proc/sys/vm/overcommit_memory to 1.

 And don't be afraid, the memory is not actually consumed by
  cmsfs-fuse,
 it is just needed to pass several checks buried in the memory
  management
 of the Linux kernel.
   
There seems to be something wrong with this approach to me.  Why
  should I
have to play around with kernel parameters just to mount a file system?
 What is the fuse driver doing that the kernel thinks it needs so much
memory?
   
   
Mark Post
   
--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390
  or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
   
  
   --
   For LINUX-390 subscribe / signoff / archive access instructions,
   send email to lists...@vm.marist.edu with the message: INFO LINUX-390
  or visit
   http://www.marist.edu/htbin/wlvindex?LINUX-390
   --
   For more information on Linux on System z, visit
   http://wiki.linuxvm.org/
  
 
  --
  For LINUX-390 subscribe / signoff / archive access instructions,
  send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
  visit
  http://www.marist.edu/htbin/wlvindex?LINUX-390
  --
  For more information on Linux on System z, visit
  http://wiki.linuxvm.org/
 
 
 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/
 

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Jan Glauber]

On Fri, 2011-12-02 at 14:05 -0600, David Boyes wrote:
  It's just mmap'ing the whole disk into the process's address space for the
  programmers sake.

 BAD idea (in the sense of 'broken as designed'). You're taxing the virtual 
 memory system in a shared-resource environment, which hurts the entire 
 environment for a little convenience in programming.

  If that turns out to be a problem we could theoretically go back to
  pread/pwrite. But I'm not sure how many users have such large CMS disks?

 Please do. Doesn't matter if it's an edge case, it shouldn't do this.

Since there seems to be collective disapproval of the requirement to
touch the memory settings for large disks I'm looking into changing
that...

I can easily replace the mmap-memcpy with pread/pwrite. Unfortunately I
see huge performance drops if I do so. Currently I'm looking why the
system call variant costs so much more than mmap.

Jan

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Rob van der Heij]

On Wed, Dec 14, 2011 at 1:20 PM, Jan Glauber j...@linux.vnet.ibm.com wrote:

 Since there seems to be collective disapproval of the requirement to
 touch the memory settings for large disks I'm looking into changing
 that...

You don't get a random sample of the impacted audience  ;-)   While
the ideal world has no limitations, no rain when you walk the dog and
good performance every day, you can't always have it. You probably do
need to document the limitation and have a useful error message. But
if it hurts, then don't do it.

I think we can agree that CMS mini disks of 20 GB is beyond common
usage and intended purpose of cmsfs. In fact, CMS will show its
limitations as well when you try to do something with it. When looking
at where to put your energy, I have a dozen other things that seem far
more significant to make the platform work...

Rob (from the peanut gallery)

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Rogério Soares]

Great News Jan!

   I'm out of office for the next 2 days, as soon i can connect i will try
it with pleasure.


TIA!

Rogério Soares


On Wed, Dec 14, 2011 at 10:20 AM, Jan Glauber j...@linux.vnet.ibm.comwrote:

 On Fri, 2011-12-02 at 14:05 -0600, David Boyes wrote:
   It's just mmap'ing the whole disk into the process's address space for
 the
   programmers sake.
 
  BAD idea (in the sense of 'broken as designed'). You're taxing the
 virtual memory system in a shared-resource environment, which hurts the
 entire environment for a little convenience in programming.
 
   If that turns out to be a problem we could theoretically go back to
   pread/pwrite. But I'm not sure how many users have such large CMS
 disks?
 
  Please do. Doesn't matter if it's an edge case, it shouldn't do this.

 Since there seems to be collective disapproval of the requirement to
 touch the memory settings for large disks I'm looking into changing
 that...

 I can easily replace the mmap-memcpy with pread/pwrite. Unfortunately I
 see huge performance drops if I do so. Currently I'm looking why the
 system call variant costs so much more than mmap.

 Jan

  --
  For LINUX-390 subscribe / signoff / archive access instructions,
  send email to lists...@vm.marist.edu with the message: INFO LINUX-390
 or visit
  http://www.marist.edu/htbin/wlvindex?LINUX-390
  --
  For more information on Linux on System z, visit
  http://wiki.linuxvm.org/
 

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
 visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Mark Post]

 On 12/14/2011 at 07:40 AM, Rob van der Heij rvdh...@gmail.com wrote: 
 I think we can agree that CMS mini disks of 20 GB is beyond common
 usage and intended purpose of cmsfs.

Is 2.3GB too much to ask?  That was my situation, and I found it rather 
irritating even then.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Mark Post]

 On 12/14/2011 at 07:20 AM, Jan Glauber j...@linux.vnet.ibm.com wrote: 

 I can easily replace the mmap-memcpy with pread/pwrite. Unfortunately I
 see huge performance drops if I do so. Currently I'm looking why the
 system call variant costs so much more than mmap.

Define huge performance drops.  I don't know that most people are going to be 
reading large files from a CMS minidisk.  They're more likely to have large 
disks with lots of reasonably sized files.  I suspect that some people are 
wanting to read very large files, but I don't know how many people, or how 
large the files are.

Would it be possible to use pread for the FST, and then mmap-memcpy the actual 
file being read?  I would find it far more understandable to need more memory 
to read a very large file than I would to just mount the file system.


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Richard Troth]

I suggest two things:
First, draw a line (this would be a somewhat arbitrary number).  If
the disk is less than (for example) 256M (roughly 350 cyls of 3390),
then continue to use mmap/memcpy.  If larger, then switch to
pread/pwrite.  Second, provide a mount option so that the user can
always specify which model to they want.  (With the option to be
explicit, there is little reason any of us can complain about where
you draw that line.)

-- R;   
Rick Troth
Velocity Software
http://www.velocitysoftware.com/





On Wed, Dec 14, 2011 at 07:20, Jan Glauber j...@linux.vnet.ibm.com wrote:
 On Fri, 2011-12-02 at 14:05 -0600, David Boyes wrote:
  It's just mmap'ing the whole disk into the process's address space for the
  programmers sake.

 BAD idea (in the sense of 'broken as designed'). You're taxing the virtual 
 memory system in a shared-resource environment, which hurts the entire 
 environment for a little convenience in programming.

  If that turns out to be a problem we could theoretically go back to
  pread/pwrite. But I'm not sure how many users have such large CMS disks?

 Please do. Doesn't matter if it's an edge case, it shouldn't do this.

 Since there seems to be collective disapproval of the requirement to
 touch the memory settings for large disks I'm looking into changing
 that...

 I can easily replace the mmap-memcpy with pread/pwrite. Unfortunately I
 see huge performance drops if I do so. Currently I'm looking why the
 system call variant costs so much more than mmap.

 Jan

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or 
 visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/


 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[David Boyes]

 I can easily replace the mmap-memcpy with pread/pwrite. Unfortunately I
 see huge performance drops if I do so. Currently I'm looking why the system
 call variant costs so much more than mmap.

An alternative approach would be to mmap *only* the minidisk label and FST on 
access of the minidisk, and then mmap individual files as you need them. I'm 
not objecting to the use of mmap, just the gratuitous use of mmap on the whole 
disk rather than what you actually *need* at that moment. 

It's like the old Antonin Chekov quote about what's necessary in a play: if 
there's a pistol on stage, you need to have killed someone by act II, scene 1. 
Otherwise, take it out -- it didn't need to be there. 

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Alan Altmark]

On Friday, 12/02/2011 at 02:56 EST, Jan Glauber j...@linux.vnet.ibm.com
wrote:
 It's just mmap'ing the whole disk into the process's address space for
 the programmers sake.

 If that turns out to be a problem we could theoretically go back to
 pread/pwrite. But I'm not sure how many users have such large CMS disks?

Hi, Jan.  I'm going to have to agree with Mark and David that the idea of
messing with memory overcommit and the ulimits in order to get a file
system to function is a non-starter.  It's too complicated.

If you want to using memory mapping, that's fine, I guess, but it needs to
map a much smaller window on the disk and move the window around as
needed.

Alan Altmark

Senior Managing z/VM and Linux Consultant
IBM System Lab Services and Training
ibm.com/systems/services/labservices
office: 607.429.3323
mobile; 607.321.7556
alan_altm...@us.ibm.com
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

cmsfs-fuse: mmap failed: Cannot allocate memory

[Rogério Soares]

Hello List, someone  got this error before?

TIA.

capp101:~ # cmsfs-fuse -t /dev/dasdc /linmon
cmsfs-fuse: mmap failed: Cannot allocate memory


/dev/dasdc is a minidisk



'On User direct guest definition:

LINK LINMON 291 291 RR


'On Linux

capp101:~ # uname -a
Linux capp101 2.6.32.12-0.7-default #1 SMP 2010-05-20 11:14:20 +0200 s390x
s390x s390x GNU/Linux

capp101:~ # cat /etc/SuSE-release
SUSE Linux Enterprise Server 11 (s390x)
VERSION = 11
PATCHLEVEL = 1
capp101:~ #


capp101:~ # /usr/bin/cmsfs-fuse -v
cmsfs-fuse: FUSE file system for CMS disks program version
1.14.0-build-20111201
Copyright IBM Corp. 2010



capp101:~ # lsdasd
Bus-ID Status  Name  Device  Type  BlkSz  Size  Blocks
==
0.0.0291   active  dasdc 94:8ECKD  4096   21121MB   5407020


capp101:~ # cmsfs-fuse -t /dev/dasdc /linmon
cmsfs-fuse: mmap failed: Cannot allocate memory

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Jan Glauber]

Hello Rogerio,

On Fri, 2011-12-02 at 13:25 -0200, Rogério Soares wrote:
 Hello List, someone  got this error before?
 
 TIA.
 
 capp101:~ # cmsfs-fuse -t /dev/dasdc /linmon
 cmsfs-fuse: mmap failed: Cannot allocate memory

That error means cmsfs-fuse fails to create the mmap of the disk into
the process's memory.

What does 'ulimit -v' show on your machine?

Jan


---
Linux on System z Kernel developer, LTC Boeblingen

IBM Deutschland Entwicklung GmbH
Vorsitzender des Aufsichtsrats: Martin Jetter Geschäftsführung: Herbert
Kircher, Sitz der Gesellschaft: Böblingen, Registergericht Amtsgericht
Stuttgart, HRB 243294

 
 /dev/dasdc is a minidisk
 
 
 
 'On User direct guest definition:
 
 LINK LINMON 291 291 RR
 
 
 'On Linux
 
 capp101:~ # uname -a
 Linux capp101 2.6.32.12-0.7-default #1 SMP 2010-05-20 11:14:20 +0200 s390x
 s390x s390x GNU/Linux
 
 capp101:~ # cat /etc/SuSE-release
 SUSE Linux Enterprise Server 11 (s390x)
 VERSION = 11
 PATCHLEVEL = 1
 capp101:~ #
 
 
 capp101:~ # /usr/bin/cmsfs-fuse -v
 cmsfs-fuse: FUSE file system for CMS disks program version
 1.14.0-build-20111201
 Copyright IBM Corp. 2010
 
 
 
 capp101:~ # lsdasd
 Bus-ID Status  Name  Device  Type  BlkSz  Size  Blocks
 ==
 0.0.0291   active  dasdc 94:8ECKD  4096   21121MB   5407020
 
 
 capp101:~ # cmsfs-fuse -t /dev/dasdc /linmon
 cmsfs-fuse: mmap failed: Cannot allocate memory
 
 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/
 

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Rogério Soares]

Hi Jan,

capp101:~ # ulimit -v
1233040




On Fri, Dec 2, 2011 at 1:56 PM, Jan Glauber j...@linux.vnet.ibm.com wrote:

 Hello Rogerio,

 On Fri, 2011-12-02 at 13:25 -0200, Rogério Soares wrote:
  Hello List, someone  got this error before?
 
  TIA.
 
  capp101:~ # cmsfs-fuse -t /dev/dasdc /linmon
  cmsfs-fuse: mmap failed: Cannot allocate memory

 That error means cmsfs-fuse fails to create the mmap of the disk into
 the process's memory.

 What does 'ulimit -v' show on your machine?

 Jan


 ---
 Linux on System z Kernel developer, LTC Boeblingen

 IBM Deutschland Entwicklung GmbH
 Vorsitzender des Aufsichtsrats: Martin Jetter Geschäftsführung: Herbert
 Kircher, Sitz der Gesellschaft: Böblingen, Registergericht Amtsgericht
 Stuttgart, HRB 243294

 
  /dev/dasdc is a minidisk
 
  
 
  'On User direct guest definition:
 
  LINK LINMON 291 291 RR
 
 
  'On Linux
 
  capp101:~ # uname -a
  Linux capp101 2.6.32.12-0.7-default #1 SMP 2010-05-20 11:14:20 +0200
 s390x
  s390x s390x GNU/Linux
 
  capp101:~ # cat /etc/SuSE-release
  SUSE Linux Enterprise Server 11 (s390x)
  VERSION = 11
  PATCHLEVEL = 1
  capp101:~ #
 
 
  capp101:~ # /usr/bin/cmsfs-fuse -v
  cmsfs-fuse: FUSE file system for CMS disks program version
  1.14.0-build-20111201
  Copyright IBM Corp. 2010
 
 
 
  capp101:~ # lsdasd
  Bus-ID Status  Name  Device  Type  BlkSz  Size  Blocks
 
 ==
  0.0.0291   active  dasdc 94:8ECKD  4096   21121MB   5407020
 
 
  capp101:~ # cmsfs-fuse -t /dev/dasdc /linmon
  cmsfs-fuse: mmap failed: Cannot allocate memory
 
  --
  For LINUX-390 subscribe / signoff / archive access instructions,
  send email to lists...@vm.marist.edu with the message: INFO LINUX-390
 or visit
  http://www.marist.edu/htbin/wlvindex?LINUX-390
  --
  For more information on Linux on System z, visit
  http://wiki.linuxvm.org/
 

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
 visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Richard Troth]

Is that really a 21GB CMS disk?
On Dec 2, 2011 10:27 AM, Rogério Soares rogerio.soa...@gmail.com wrote:

 Hello List, someone  got this error before?

 TIA.

 capp101:~ # cmsfs-fuse -t /dev/dasdc /linmon
 cmsfs-fuse: mmap failed: Cannot allocate memory


 /dev/dasdc is a minidisk

 

 'On User direct guest definition:

 LINK LINMON 291 291 RR


 'On Linux

 capp101:~ # uname -a
 Linux capp101 2.6.32.12-0.7-default #1 SMP 2010-05-20 11:14:20 +0200 s390x
 s390x s390x GNU/Linux

 capp101:~ # cat /etc/SuSE-release
 SUSE Linux Enterprise Server 11 (s390x)
 VERSION = 11
 PATCHLEVEL = 1
 capp101:~ #


 capp101:~ # /usr/bin/cmsfs-fuse -v
 cmsfs-fuse: FUSE file system for CMS disks program version
 1.14.0-build-20111201
 Copyright IBM Corp. 2010



 capp101:~ # lsdasd
 Bus-ID Status  Name  Device  Type  BlkSz  Size  Blocks

 ==
 0.0.0291   active  dasdc 94:8ECKD  4096   21121MB   5407020


 capp101:~ # cmsfs-fuse -t /dev/dasdc /linmon
 cmsfs-fuse: mmap failed: Cannot allocate memory

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
 visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Jan Glauber]

On Fri, 2011-12-02 at 14:21 -0200, Rogério Soares wrote:
 Hi Jan,
 
 capp101:~ # ulimit -v
 1233040
 

Try to increase the limit, it should be bigger than the size of the disk
you're trying to mount (or unlimited).

If you don't have enough free memory in your guest you may also need to
set  /proc/sys/vm/overcommit_memory to 1.

And don't be afraid, the memory is not actually consumed by cmsfs-fuse,
it is just needed to pass several checks buried in the memory management
of the Linux kernel.

Jan

 
 
 On Fri, Dec 2, 2011 at 1:56 PM, Jan Glauber j...@linux.vnet.ibm.com wrote:
 
  Hello Rogerio,
 
  On Fri, 2011-12-02 at 13:25 -0200, Rogério Soares wrote:
   Hello List, someone  got this error before?
  
   TIA.
  
   capp101:~ # cmsfs-fuse -t /dev/dasdc /linmon
   cmsfs-fuse: mmap failed: Cannot allocate memory
 
  That error means cmsfs-fuse fails to create the mmap of the disk into
  the process's memory.
 
  What does 'ulimit -v' show on your machine?
 
  Jan
 
 
  ---
  Linux on System z Kernel developer, LTC Boeblingen
 
  IBM Deutschland Entwicklung GmbH
  Vorsitzender des Aufsichtsrats: Martin Jetter Geschäftsführung: Herbert
  Kircher, Sitz der Gesellschaft: Böblingen, Registergericht Amtsgericht
  Stuttgart, HRB 243294
 
  
   /dev/dasdc is a minidisk
  
   
  
   'On User direct guest definition:
  
   LINK LINMON 291 291 RR
  
  
   'On Linux
  
   capp101:~ # uname -a
   Linux capp101 2.6.32.12-0.7-default #1 SMP 2010-05-20 11:14:20 +0200
  s390x
   s390x s390x GNU/Linux
  
   capp101:~ # cat /etc/SuSE-release
   SUSE Linux Enterprise Server 11 (s390x)
   VERSION = 11
   PATCHLEVEL = 1
   capp101:~ #
  
  
   capp101:~ # /usr/bin/cmsfs-fuse -v
   cmsfs-fuse: FUSE file system for CMS disks program version
   1.14.0-build-20111201
   Copyright IBM Corp. 2010
  
  
  
   capp101:~ # lsdasd
   Bus-ID Status  Name  Device  Type  BlkSz  Size  Blocks
  
  ==
   0.0.0291   active  dasdc 94:8ECKD  4096   21121MB   5407020
  
  
   capp101:~ # cmsfs-fuse -t /dev/dasdc /linmon
   cmsfs-fuse: mmap failed: Cannot allocate memory
  
   --
   For LINUX-390 subscribe / signoff / archive access instructions,
   send email to lists...@vm.marist.edu with the message: INFO LINUX-390
  or visit
   http://www.marist.edu/htbin/wlvindex?LINUX-390
   --
   For more information on Linux on System z, visit
   http://wiki.linuxvm.org/
  
 
  --
  For LINUX-390 subscribe / signoff / archive access instructions,
  send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
  visit
  http://www.marist.edu/htbin/wlvindex?LINUX-390
  --
  For more information on Linux on System z, visit
  http://wiki.linuxvm.org/
 
 
 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/
 

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Rogério Soares]

Yes Richard, this disk save old logs monitoring... :)



On Fri, Dec 2, 2011 at 3:32 PM, Richard Troth vmcow...@gmail.com wrote:

 Is that really a 21GB CMS disk?
 On Dec 2, 2011 10:27 AM, Rogério Soares rogerio.soa...@gmail.com
 wrote:

  Hello List, someone  got this error before?
 
  TIA.
 
  capp101:~ # cmsfs-fuse -t /dev/dasdc /linmon
  cmsfs-fuse: mmap failed: Cannot allocate memory
 
 
  /dev/dasdc is a minidisk
 
  
 
  'On User direct guest definition:
 
  LINK LINMON 291 291 RR
 
 
  'On Linux
 
  capp101:~ # uname -a
  Linux capp101 2.6.32.12-0.7-default #1 SMP 2010-05-20 11:14:20 +0200
 s390x
  s390x s390x GNU/Linux
 
  capp101:~ # cat /etc/SuSE-release
  SUSE Linux Enterprise Server 11 (s390x)
  VERSION = 11
  PATCHLEVEL = 1
  capp101:~ #
 
 
  capp101:~ # /usr/bin/cmsfs-fuse -v
  cmsfs-fuse: FUSE file system for CMS disks program version
  1.14.0-build-20111201
  Copyright IBM Corp. 2010
 
 
 
  capp101:~ # lsdasd
  Bus-ID Status  Name  Device  Type  BlkSz  Size  Blocks
 
 
 ==
  0.0.0291   active  dasdc 94:8ECKD  4096   21121MB   5407020
 
 
  capp101:~ # cmsfs-fuse -t /dev/dasdc /linmon
  cmsfs-fuse: mmap failed: Cannot allocate memory
 
  --
  For LINUX-390 subscribe / signoff / archive access instructions,
  send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
  visit
  http://www.marist.edu/htbin/wlvindex?LINUX-390
  --
  For more information on Linux on System z, visit
  http://wiki.linuxvm.org/
 

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
 visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Rogério Soares]

Works Like a charm Jan!!

df -h

/dev/dasdc 21G   15G  6.1G  71% /linmon


THANK YOU! :)


On Fri, Dec 2, 2011 at 3:26 PM, Jan Glauber j...@linux.vnet.ibm.com wrote:

 On Fri, 2011-12-02 at 14:21 -0200, Rogério Soares wrote:
  Hi Jan,
 
  capp101:~ # ulimit -v
  1233040
 

 Try to increase the limit, it should be bigger than the size of the disk
 you're trying to mount (or unlimited).

 If you don't have enough free memory in your guest you may also need to
 set  /proc/sys/vm/overcommit_memory to 1.

 And don't be afraid, the memory is not actually consumed by cmsfs-fuse,
 it is just needed to pass several checks buried in the memory management
 of the Linux kernel.

 Jan

 
 
  On Fri, Dec 2, 2011 at 1:56 PM, Jan Glauber j...@linux.vnet.ibm.com
 wrote:
 
   Hello Rogerio,
  
   On Fri, 2011-12-02 at 13:25 -0200, Rogério Soares wrote:
Hello List, someone  got this error before?
   
TIA.
   
capp101:~ # cmsfs-fuse -t /dev/dasdc /linmon
cmsfs-fuse: mmap failed: Cannot allocate memory
  
   That error means cmsfs-fuse fails to create the mmap of the disk into
   the process's memory.
  
   What does 'ulimit -v' show on your machine?
  
   Jan
  
  
   ---
   Linux on System z Kernel developer, LTC Boeblingen
  
   IBM Deutschland Entwicklung GmbH
   Vorsitzender des Aufsichtsrats: Martin Jetter Geschäftsführung: Herbert
   Kircher, Sitz der Gesellschaft: Böblingen, Registergericht Amtsgericht
   Stuttgart, HRB 243294
  
   
/dev/dasdc is a minidisk
   

   
'On User direct guest definition:
   
LINK LINMON 291 291 RR
   
   
'On Linux
   
capp101:~ # uname -a
Linux capp101 2.6.32.12-0.7-default #1 SMP 2010-05-20 11:14:20 +0200
   s390x
s390x s390x GNU/Linux
   
capp101:~ # cat /etc/SuSE-release
SUSE Linux Enterprise Server 11 (s390x)
VERSION = 11
PATCHLEVEL = 1
capp101:~ #
   
   
capp101:~ # /usr/bin/cmsfs-fuse -v
cmsfs-fuse: FUSE file system for CMS disks program version
1.14.0-build-20111201
Copyright IBM Corp. 2010
   
   
   
capp101:~ # lsdasd
Bus-ID Status  Name  Device  Type  BlkSz  Size
  Blocks
   
  
 ==
0.0.0291   active  dasdc 94:8ECKD  4096   21121MB
 5407020
   
   
capp101:~ # cmsfs-fuse -t /dev/dasdc /linmon
cmsfs-fuse: mmap failed: Cannot allocate memory
   
   
 --
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO
 LINUX-390
   or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
   
 --
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
   
  
   --
   For LINUX-390 subscribe / signoff / archive access instructions,
   send email to lists...@vm.marist.edu with the message: INFO LINUX-390
 or
   visit
   http://www.marist.edu/htbin/wlvindex?LINUX-390
   --
   For more information on Linux on System z, visit
   http://wiki.linuxvm.org/
  
 
  --
  For LINUX-390 subscribe / signoff / archive access instructions,
  send email to lists...@vm.marist.edu with the message: INFO LINUX-390
 or visit
  http://www.marist.edu/htbin/wlvindex?LINUX-390
  --
  For more information on Linux on System z, visit
  http://wiki.linuxvm.org/
 

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
 visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Mark Post]

 On 12/2/2011 at 12:26 PM, Jan Glauber j...@linux.vnet.ibm.com wrote: 
 Try to increase the limit, it should be bigger than the size of the disk
 you're trying to mount (or unlimited).
 
 If you don't have enough free memory in your guest you may also need to
 set  /proc/sys/vm/overcommit_memory to 1.
 
 And don't be afraid, the memory is not actually consumed by cmsfs-fuse,
 it is just needed to pass several checks buried in the memory management
 of the Linux kernel.

There seems to be something wrong with this approach to me.  Why should I have 
to play around with kernel parameters just to mount a file system?  What is the 
fuse driver doing that the kernel thinks it needs so much memory?


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Jan Glauber]

On Fri, 2011-12-02 at 16:36 -0200, Rogério Soares wrote:
 Jan,
 
 i had mount using:
 
 cmsfs-fuse /dev/dasdc /linmon/ -o allow_root
 
 capp101:/linmon # cmsfs-fuse /dev/dasdc /linmon/ -o allow_root
 
 when, i tried copy files to another location, i got:
 
 capp101:/linmon # cp Z196PROD.MD000675 /root/
 
 cp: cannot open `Z196PROD.MD000675' for reading: Input/output error
 
 is it supposed to be ok?

Well, seriously not. This looks like a bug or a broken file.

Can you access the file under CMS? Does this happen for every file on
the disk or only for this file?

Jan

 
 
 On Fri, Dec 2, 2011 at 4:19 PM, Mark Post mp...@novell.com wrote:
 
   On 12/2/2011 at 12:26 PM, Jan Glauber j...@linux.vnet.ibm.com wrote:
   Try to increase the limit, it should be bigger than the size of the disk
   you're trying to mount (or unlimited).
  
   If you don't have enough free memory in your guest you may also need to
   set  /proc/sys/vm/overcommit_memory to 1.
  
   And don't be afraid, the memory is not actually consumed by cmsfs-fuse,
   it is just needed to pass several checks buried in the memory management
   of the Linux kernel.
 
  There seems to be something wrong with this approach to me.  Why should I
  have to play around with kernel parameters just to mount a file system?
   What is the fuse driver doing that the kernel thinks it needs so much
  memory?
 
 
  Mark Post
 
  --
  For LINUX-390 subscribe / signoff / archive access instructions,
  send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
  visit
  http://www.marist.edu/htbin/wlvindex?LINUX-390
  --
  For more information on Linux on System z, visit
  http://wiki.linuxvm.org/
 
 
 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/
 

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[David Boyes]

 It's just mmap'ing the whole disk into the process's address space for the
 programmers sake.

BAD idea (in the sense of 'broken as designed'). You're taxing the virtual 
memory system in a shared-resource environment, which hurts the entire 
environment for a little convenience in programming. 
 
 If that turns out to be a problem we could theoretically go back to
 pread/pwrite. But I'm not sure how many users have such large CMS disks?

Please do. Doesn't matter if it's an edge case, it shouldn't do this. 

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Rogério Soares]

under cms without problem apparently, i made a loop for try copy, the most
files i can´t copy ... :-/


On Fri, Dec 2, 2011 at 6:00 PM, Jan Glauber j...@linux.vnet.ibm.com wrote:

 On Fri, 2011-12-02 at 16:36 -0200, Rogério Soares wrote:
  Jan,
 
  i had mount using:
 
  cmsfs-fuse /dev/dasdc /linmon/ -o allow_root
 
  capp101:/linmon # cmsfs-fuse /dev/dasdc /linmon/ -o allow_root
 
  when, i tried copy files to another location, i got:
 
  capp101:/linmon # cp Z196PROD.MD000675 /root/
 
  cp: cannot open `Z196PROD.MD000675' for reading: Input/output error
 
  is it supposed to be ok?

 Well, seriously not. This looks like a bug or a broken file.

 Can you access the file under CMS? Does this happen for every file on
 the disk or only for this file?

 Jan

 
 
  On Fri, Dec 2, 2011 at 4:19 PM, Mark Post mp...@novell.com wrote:
 
On 12/2/2011 at 12:26 PM, Jan Glauber j...@linux.vnet.ibm.com
 wrote:
Try to increase the limit, it should be bigger than the size of the
 disk
you're trying to mount (or unlimited).
   
If you don't have enough free memory in your guest you may also need
 to
set  /proc/sys/vm/overcommit_memory to 1.
   
And don't be afraid, the memory is not actually consumed by
 cmsfs-fuse,
it is just needed to pass several checks buried in the memory
 management
of the Linux kernel.
  
   There seems to be something wrong with this approach to me.  Why
 should I
   have to play around with kernel parameters just to mount a file system?
What is the fuse driver doing that the kernel thinks it needs so much
   memory?
  
  
   Mark Post
  
   --
   For LINUX-390 subscribe / signoff / archive access instructions,
   send email to lists...@vm.marist.edu with the message: INFO LINUX-390
 or
   visit
   http://www.marist.edu/htbin/wlvindex?LINUX-390
   --
   For more information on Linux on System z, visit
   http://wiki.linuxvm.org/
  
 
  --
  For LINUX-390 subscribe / signoff / archive access instructions,
  send email to lists...@vm.marist.edu with the message: INFO LINUX-390
 or visit
  http://www.marist.edu/htbin/wlvindex?LINUX-390
  --
  For more information on Linux on System z, visit
  http://wiki.linuxvm.org/
 

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
 visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Jan Glauber]

On Fri, 2011-12-02 at 14:05 -0600, David Boyes wrote:
  It's just mmap'ing the whole disk into the process's address space for the
  programmers sake.

 BAD idea (in the sense of 'broken as designed'). You're taxing the virtual 
 memory system in a shared-resource environment, which hurts the entire 
 environment for a little convenience in programming.

David, can you explain me how I'm taxing the VM? The mmap operation
does not allocate a single physical page. If I remember right it does
not even set up the page tables for the mapping.

Jan

  If that turns out to be a problem we could theoretically go back to
  pread/pwrite. But I'm not sure how many users have such large CMS disks?

 Please do. Doesn't matter if it's an edge case, it shouldn't do this.

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Jan Glauber]

On Fri, 2011-12-02 at 18:10 -0200, Rogério Soares wrote:
 under cms without problem apparently, i made a loop for try copy, the most
 files i can´t copy ... :-/

Looks like I need to blame myself :/

I've made several bug fixes which could possibly solve the problem
you're seeing but the s390-tools update that contains these fixes is not
yet released.

Feel free to open a bug report!

Jan

 
 On Fri, Dec 2, 2011 at 6:00 PM, Jan Glauber j...@linux.vnet.ibm.com wrote:
 
  On Fri, 2011-12-02 at 16:36 -0200, Rogério Soares wrote:
   Jan,
  
   i had mount using:
  
   cmsfs-fuse /dev/dasdc /linmon/ -o allow_root
  
   capp101:/linmon # cmsfs-fuse /dev/dasdc /linmon/ -o allow_root
  
   when, i tried copy files to another location, i got:
  
   capp101:/linmon # cp Z196PROD.MD000675 /root/
  
   cp: cannot open `Z196PROD.MD000675' for reading: Input/output error
  
   is it supposed to be ok?
 
  Well, seriously not. This looks like a bug or a broken file.
 
  Can you access the file under CMS? Does this happen for every file on
  the disk or only for this file?
 
  Jan
 
  
  
   On Fri, Dec 2, 2011 at 4:19 PM, Mark Post mp...@novell.com wrote:
  
 On 12/2/2011 at 12:26 PM, Jan Glauber j...@linux.vnet.ibm.com
  wrote:
 Try to increase the limit, it should be bigger than the size of the
  disk
 you're trying to mount (or unlimited).

 If you don't have enough free memory in your guest you may also need
  to
 set  /proc/sys/vm/overcommit_memory to 1.

 And don't be afraid, the memory is not actually consumed by
  cmsfs-fuse,
 it is just needed to pass several checks buried in the memory
  management
 of the Linux kernel.
   
There seems to be something wrong with this approach to me.  Why
  should I
have to play around with kernel parameters just to mount a file system?
 What is the fuse driver doing that the kernel thinks it needs so much
memory?
   
   
Mark Post
   
--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390
  or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
   
  
   --
   For LINUX-390 subscribe / signoff / archive access instructions,
   send email to lists...@vm.marist.edu with the message: INFO LINUX-390
  or visit
   http://www.marist.edu/htbin/wlvindex?LINUX-390
   --
   For more information on Linux on System z, visit
   http://wiki.linuxvm.org/
  
 
  --
  For LINUX-390 subscribe / signoff / archive access instructions,
  send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
  visit
  http://www.marist.edu/htbin/wlvindex?LINUX-390
  --
  For more information on Linux on System z, visit
  http://wiki.linuxvm.org/
 
 
 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/
 

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[Richard Troth]

(Butting in for David just to be annoying!)

I bet he means FUSE is taxing the shared environment by requiring such
a large chunk of memory to be mapped.  Even if sparse, tables for
managing it will be needed.

-- R;   
Rick Troth
Velocity Software
http://www.velocitysoftware.com/





On Fri, Dec 2, 2011 at 15:17, Jan Glauber j...@linux.vnet.ibm.com wrote:
 On Fri, 2011-12-02 at 14:05 -0600, David Boyes wrote:
  It's just mmap'ing the whole disk into the process's address space for the
  programmers sake.

 BAD idea (in the sense of 'broken as designed'). You're taxing the virtual 
 memory system in a shared-resource environment, which hurts the entire 
 environment for a little convenience in programming.

 David, can you explain me how I'm taxing the VM? The mmap operation
 does not allocate a single physical page. If I remember right it does
 not even set up the page tables for the mapping.

 Jan

  If that turns out to be a problem we could theoretically go back to
  pread/pwrite. But I'm not sure how many users have such large CMS disks?

 Please do. Doesn't matter if it's an edge case, it shouldn't do this.

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or 
 visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/


 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse: mmap failed: Cannot allocate memory

[David Boyes]

 David, can you explain me how I'm taxing the VM? The mmap operation
 does not allocate a single physical page. If I remember right it does not even
 set up the page tables for the mapping.

Depends on which kernel release you're on. A process shouldn't need that kind 
of processing just to determine if a disk can be accessed. It seems to be 
fundamentally wrong to ask for more RAM than it takes to read the FST on 
access. 

Mmaping a file when reading it makes sense, mmaping the entire disk does not. 

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

cmsfs-fuse package in RHEL 6.1

[Brad Hinson]

 Hi,

Just an fyi for anyone interested in cmsfs.

cmsfs-fuse is a new feature included in RHEL 6.1 (released last week)
which allows write access to a CMS filesystem.  For example, it's now
possible to edit files on a guest's 191 disk with a Linux text editor
like 'vi'.

It's contained in the package 's390utils-cmsfs-fuse', but you won't find
it in the normal channels for download with RHEL 6.1 on
http://access.redhat.com.  Instead, it's in the 'optional' channel for
6.1.  For ease, here's a link:

https://rhn.redhat.com/rhn/software/packages/details/Overview.do?pid=631604
(redhat.com login required)

Some new packages land here by default, but you may see it merged into
the normal channels by RHEL 6.2.  Please let me know if anyone has
trouble getting this new package.

Thanks,
-Brad

--
Brad Hinsonbhin...@redhat.com
Worldwide System z Sales, Strategy, Marketing
Red Hat, Inc.
+1 (919) 360-0443 (mobile)
www.redhat.com/z

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse package in RHEL 6.1

[Shane G]

H - and why isn't this generally available ?.
Open source ... ???

Not meaning to piss on the messenger, but I'm happy to do likewise to the
policy makers involved.
This is just *bad*.

Put it out there, let the community benefit.
Having been the recipient of IBMs progression to OCO, this just rubs me the
wrong way.

Shane ...

 Just an fyi for anyone interested in cmsfs.
 
 cmsfs-fuse is a new feature included in RHEL 6.1 (released last week)
 which allows write access to a CMS filesystem.  For example, it's now
 possible to edit files on a guest's 191 disk with a Linux text editor
 like 'vi'.
 
 It's contained in the package 's390utils-cmsfs-fuse', but you won't find
 it in the normal channels for download with RHEL 6.1 on
 http://access.redhat.com.  Instead, it's in the 'optional' channel for
 6.1.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse package in RHEL 6.1

[Philipp Kern]

On Sat, May 28, 2011 at 10:21:04PM +1000, Shane G wrote:
 H - and why isn't this generally available ?.
 Open source ... ???
 
 Not meaning to piss on the messenger, but I'm happy to do likewise to the
 policy makers involved.
 This is just *bad*.
 
 Put it out there, let the community benefit.
 Having been the recipient of IBMs progression to OCO, this just rubs me the
 wrong way.

Meh, this errata[0] tells me that it's in the s390utils SRPM and mighty Google
tells me that it's available on [1].

They could make it easier, though.  But then it's in s390-utils upstream
anyway: [2]

Kind regards,
Philipp Kern

[0] http://rhn.redhat.com/errata/RHBA-2011-0601.html
[1] 
http://www.jur-linux.com/download/ftp.redhat.com/enterprise/6Server/en/os/SRPMS/s390utils-1.8.2-36.el6.src.rpm
[2] http://www.ibm.com/developerworks/linux/linux390/s390-tools-1.13.0.html

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/


signature.asc
Description: Digital signature

Re: cmsfs-fuse package in RHEL 6.1

[Scott Rohling]

http://www.ibm.com/developerworks/linux/linux390/s390-tools-1.9.0.html

http://www.ibm.com/developerworks/linux/linux390/s390-tools-1.9.0.htmlDoes
this help you see it as *good* ?

Scott Rohling

On Sat, May 28, 2011 at 6:21 AM, Shane G ibm-m...@tpg.com.au wrote:

 H - and why isn't this generally available ?.
 Open source ... ???

 Not meaning to piss on the messenger, but I'm happy to do likewise to the
 policy makers involved.
 This is just *bad*.

 Put it out there, let the community benefit.
 Having been the recipient of IBMs progression to OCO, this just rubs me
 the
 wrong way.

 Shane ...

  Just an fyi for anyone interested in cmsfs.
 
  cmsfs-fuse is a new feature included in RHEL 6.1 (released last week)
  which allows write access to a CMS filesystem.  For example, it's now
  possible to edit files on a guest's 191 disk with a Linux text editor
  like 'vi'.
 
  It's contained in the package 's390utils-cmsfs-fuse', but you won't find
  it in the normal channels for download with RHEL 6.1 on
  http://access.redhat.com.  Instead, it's in the 'optional' channel for
  6.1.

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
 visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse package in RHEL 6.1

[Scott Rohling]

Oops - thanks Phillipp, I was pointing to older release at developerworks --
1.13.0 is the latest, so your url is better..

Scott Rohling

On Sat, May 28, 2011 at 6:31 AM, Philipp Kern pk...@debian.org wrote:

 On Sat, May 28, 2011 at 10:21:04PM +1000, Shane G wrote:
  H - and why isn't this generally available ?.
  Open source ... ???
 
  Not meaning to piss on the messenger, but I'm happy to do likewise to the
  policy makers involved.
  This is just *bad*.
 
  Put it out there, let the community benefit.
  Having been the recipient of IBMs progression to OCO, this just rubs me
 the
  wrong way.

 Meh, this errata[0] tells me that it's in the s390utils SRPM and mighty
 Google
 tells me that it's available on [1].

 They could make it easier, though.  But then it's in s390-utils upstream
 anyway: [2]

 Kind regards,
 Philipp Kern

 [0] http://rhn.redhat.com/errata/RHBA-2011-0601.html
 [1]
 http://www.jur-linux.com/download/ftp.redhat.com/enterprise/6Server/en/os/SRPMS/s390utils-1.8.2-36.el6.src.rpm
 [2]
 http://www.ibm.com/developerworks/linux/linux390/s390-tools-1.13.0.html

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
 visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/

 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.10 (GNU/Linux)

 iEYEAREIAAYFAk3g6zYACgkQ7Ro5M7LPzdjP8QCgqth6/YRn68svhe2ThF5Ii5DC
 hYYAnRa216oA66C6GT7TiH72NvusjS7y
 =vGJk
 -END PGP SIGNATURE-



--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse package in RHEL 6.1

[Shane G]

 ... which allows write access to a CMS filesystem.

Note the *write* access - is this generally available ?.

Shane ...

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: cmsfs-fuse package in RHEL 6.1

[Scott Rohling]

It is from IBM ..  here's a helpful overview of s390-tools .. and indicates
CMS write was added in s390-tools v1.11.0:

http://www.ibm.com/developerworks/linux/linux390/s390-tools-overview.html

http://www.ibm.com/developerworks/linux/linux390/s390-tools-overview.htmlScott
Rohling

On Sat, May 28, 2011 at 6:49 AM, Shane G ibm-m...@tpg.com.au wrote:

  ... which allows write access to a CMS filesystem.

 Note the *write* access - is this generally available ?.

 Shane ...

 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
 visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Sample REXX with cmsfs usage

[Bruce Furber]

The clone.sh script in the cookbook uses CP commands and read CMS Files.
http://www.redbooks.ibm.com/redbooks/pdfs/sg247493.pdf

Bruce
- Original Message -
From: Lionel Dyck ld...@us.ibm.com
To: LINUX-390@VM.MARIST.EDU
Sent: Tuesday, August 18, 2009 2:01 PM
Subject: Sample REXX with cmsfs usage


I've been playing with rexx on z/linux along with trying to learn how to
use the cmsfs toolset. Here is my first attempt that may be useful for
others.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390

Re: Sample REXX with cmsfs usage

[Shedlock, George]

Here is another method to find a device number:

EnableADisk: /* Enable my 191 (A) disk*/
'chccwdev -e 191  /dev/null 21'
If RC  0 Then Do
Call LogCmd 'Unable to enable 191 disk. RC from chccwdev =' RC
Return 2
End
'cat /proc/dasd/devices | grep 0.0.0191 | rxqueue'
Parse Pull rec
Adisk = '/dev/' || word(rec,7)
Call LogCmd Adisk is: Adisk
Return

Here are a couple of other options for reading a file:

/* Read a standard linux file  */
'cat /etc/HOSTNAME | rxqueue'
Parse UPPER Pull rec

Another one:

GetDefaultConfig:/* Read the default config file from the 191 disk  */
TargetParm = 'DEFAULT.CONFIG'
'cmsfslst -d' Adisk '| grep DEFAULT | grep CONFIG  /dev/null'
If RC  0 Then Do
Call LogCmd TargetParm not found on 191 minidisk. Ignoring
Return
End

'cmsfscat -a -d' Adisk TargetParm 'grep ^.| cut -f2 -d. | rxqueue'
Do Queued()
Parse Pull rec
/*  Do something useful with the record  */
Call LogCmd 'DefaultConfig:' rec  
End
Return




George Shedlock Jr
AEGON Information Technology
AEGON USA
502-560-3541
-Original Message-
From: Linux on 390 Port [mailto:linux-...@vm.marist.edu] On Behalf Of Steffen 
Maier
Sent: Wednesday, August 19, 2009 2:40 PM
To: LINUX-390@VM.MARIST.EDU
Subject: Re: Sample REXX with cmsfs usage

On 08/18/2009 08:01 PM, Lionel Dyck wrote:
 I've been playing with rexx on z/linux along with trying to learn how to
 use the cmsfs toolset. Here is my first attempt that may be useful for
 others.

Nice one.

 #!/usr/bin/rexx
 /* REXX routine to read a cms file from the Linux Guest 191 Disk */

 /* Mount the users 191 disk */
 'chccwdev -e 191'
 
 /* sleep 2 seconds to wait for the mount */
 call syssleep(2)

Waiting for the udev event queue to run empty is a safer way to wait for
the device to become online and available to Linux.
Depending on the version of your Linux you may use:
'udevadm settle'
or
'udevsettle'

 file = '/proc/dasd/devices'
 
 do forever
line = strip(linein(file))
if length(line) = 0 then leave
parse value line with '0.0.'addr'(' x ' is 'disk  .
if addr = '0191' then leave
end

Not sure, but this looks like an endless loop with busy waiting,
possibly eating unnecessary CPU cycles.
Chccwdev already loops (CPU friendly) to see if the device comes online
within some timeout and finally reports the outcome with an exit
errorlevel, that you may evaluate above to decide on continuation.

 'chccwdev -d 191'

Here you could do udevadm settle again just to be sure that no events
caused by setting the device offline are pending when your script exits.

My 2ct,
Steffen

Linux on System z Development

IBM Deutschland Research  Development GmbH
Vorsitzender des Aufsichtsrats: Martin Jetter
Geschäftsführung: Erich Baier
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390