We have spectrum archive with encryption on disk and tape. We get maybe a
100 or so messages like this daily. It would be nice if message had some
information about which client is the issue.
We have had client certs expire in the past. The root cause of the outage
was a network outage...iirc
>
> Fred
> __
> Fred Stock | IBM Pittsburgh Lab | 720-430-8821
> sto...@us.ibm.com
>
>
>
> ----- Original message -
> From: "J. Eric Wonderley"
> Sent by: gpfsug-discuss-boun...@spectrumscale.org
> To: gpfsug ma
I have filesets setup in a filesystem...looks like:
[root@cl005 ~]# mmlsfileset home -L
Filesets in file system 'home':
NameId RootInode ParentId Created
InodeSpace MaxInodesAllocInodes Comment
root 0 3
mmlspool might suggest there's only 1 system pool per cluster. We have 2
clusters and it has id=0 on both.
One of our clusters has 2 filesystems that have same id for two different
dataonly pools:
[root@cl001 ~]# mmlspool home all
NameId
system 0
fc_8T65537
fc_ss
Multihoming is accomplished by using subnets...see mmchconfig.
Failover networks on the other hand are not allowed. Bad network behavior
is dealt with by expelling nodes. You must have decent/supported network
gear...we have learned that lesson the hard way
On Fri, Oct 26, 2018 at 10:37 AM Luka
Hi Stephan:
I think every node in C1 and in C2 have to see every node in the server
cluster NSD-[AD].
We have a 10 node server cluster where 2 nodes do nothing but server out
nfs. Since these two are apart of the server cluster...client clusters
wanting to mount the server cluster via gpfs need
Hi Gareth:
We have the spectrum archive product with encryption. It encrypts data on
disk and tape...but not metadata. We originally had hoped to write small
files with metadata...that does not happen with encryption.
My guess is that the system pool(where metadata lives) cannot be encrypted.
Ethernet based because of the mention of Juniper.
>
> Are you attempting to do RoCE or just plain TCP/IP?
>
>
> On December 20, 2017 at 14:40:48 EST, J. Eric Wonderley <
> eric.wonder...@vt.edu> wrote:
>
> Hello:
>
> Does anyone have this type of config?
>
>
Hello:
Does anyone have this type of config?
The host configuration looks sane but we seem to observe link-down on all
mlx adapters no matter what we do.
Big picture is that we are attempting to do mc(multichassis)-lags to a core
switch. I'm somewhat fearful as to how this is implemented in the
Hello:
If I recall correctly this does not work...correct? I think the last time
I attempted this was gpfs version <=4.1. I think I attempted to add a
quorum node.
The process was that I remember doing was mmshutdown -a, mmchcluster
--ccr-disable, mmaddnode yadayada, mmchcluster --ccr-enable, m
We have several avago/lsi 9305-16e that I believe came from Advanced HPC.
Can someone recommend a another reseller of these hbas or a contact with
Advance HPC?
___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/
gt;
> NameId RootInode ParentId
> Created InodeSpace MaxInodesAllocInodes
> Comment
>
> setquotafoo2518408295 0 Thu Aug 17
> 15:17:18 201700 0
>
>
, 2017 at 9:43 AM, Edward Wahl wrote:
> On Fri, 4 Aug 2017 01:02:22 -0400
> "J. Eric Wonderley" wrote:
>
> > 4.2.2.3
> >
> > I want to think maybe this started after expanding inode space
>
> What does 'mmlsfileset home nathanfootest -L&#
i actually hit this assert and turned it in to support on this version:
Build branch "4.2.2.3 efix6 (987197)".
i was told do to exactly what sven mentioned.
i thought it strange that i did NOT hit the assert in a no pass but hit it
in a yes pass.
On Thu, Aug 3, 2017 at 9:06 AM, Sven Oehme wrote
4.2.2.3
I want to think maybe this started after expanding inode space
On Thu, Aug 3, 2017 at 9:11 AM, James Davis wrote:
> Hey,
>
> Hmm, your invocation looks valid to me. What's your GPFS level?
>
> Cheers,
>
> Jamie
>
>
> ----- Original message -
>
for one of our home filesystem we get:
mmsetquota home:nathanfootest --block 10T:10T --files 10M:10M
tssetquota: Could not get id of fileset 'nathanfootest' error (22):
'Invalid argument'.
mmedquota -j home:nathanfootest
does work however
___
gpfsug-d
No guarantee...unless you are using ess/gss solution.
Crappy network will get you loads of expels and occasional fscks. Which I
guess beats data loss and recovery from backup.
YOu probably have a network issue...they can be subtle. Gpfs is a very
extremely thorough network tester.
Eric
On We
Hi Renar:
What does 'mmlsquota -j fileset filesystem' report?
I did not think you would get a grace period of none unless the
hardlimit=softlimit.
On Mon, Jul 31, 2017 at 1:44 PM, Grunenberg, Renar <
renar.grunenb...@huk-coburg.de> wrote:
> Hallo All,
> we are on Version 4.2.3.2 and see some mi
These type messages repeat often in our logs:
017-06-20_09:25:13.676-0400: [E]
An%20attempt%20to%20send%20notification%20to%20the%20GUI%20subsystem%20failed%2E%20response%3Dcurl%3A%20%287%29%20Failed%20connect%20to%20arproto2%2Ear%2Enis%2Eisb%2Einternal%3A443%3B%20Connection%20refused%20rc%3D7
rc=
Hi Jamie:
I think typically you want to keep the clients ahead of the server in
version. I would advance the version of you client nodes.
New clients can communicate with older versions of server nsds. Vice
versa...no so much.
___
gpfsug-discuss maili
1) You want more that one quorum node on your server cluster. The
non-quorum node does need a daemon network interface exposed to the client
cluster as does the quorum nodes.
2) No. Admin network is for intra cluster communications...not inter
cluster(between clusters). Daemon interface(port
apshot gpfs | wc -l
>
> 6916
>
>
>
> *From:* gpfsug-discuss-boun...@spectrumscale.org [mailto:gpfsug-discuss-
> boun...@spectrumscale.org] *On Behalf Of *J. Eric Wonderley
> *Sent:* 20 March 2017 14:03
> *To:* gpfsug main discussion list
> *Subject:* [gpfsug-discuss] snapshot
I found this link and it didn't give me much hope for doing snapshots &
backup in a home(busy) filesystem:
http://www.spectrumscale.org/pipermail/gpfsug-discuss/2013-
February/000200.html
I realize this is dated and I wondered if qos etc have made is a tolerable
thing to do now. Gpfs I think was
al Parallel File
> System Dev.
> Pittsburgh, PA (412) 667-6993 Tie-Line 989-6993
>sber...@us.ibm.com
> Every once in a while, it is a good idea to call out, "Computer, end
> program!" just to check. --David Noelle
> All Your Bas
Anyone have any examples of this? I have a filesystem that has 2 pools and
several filesets and would like daily progressive incremental backups of
its contents.
I found some stuff here(nothing real close to what I wanted however):
/usr/lpp/mmfs/samples/ilm
I have the tsm client installed on the
Well we got it into the down state using mmsdrrestore -p to recover stuff
into /var/mmfs/gen to cl004.
Anyhow we ended up unknown for cl004 when it powered off. Short of
removing node, unknown is the state you get.
Unknown seems stable for a hopefully short outage of cl004.
Thanks
On Thu, Feb
Is there a way to accomplish this so the rest of cluster knows its down?
My state now:
[root@cl001 ~]# mmgetstate -aL
cl004.cl.arc.internal: mmremote: determineMode: Missing file
/var/mmfs/gen/mmsdrfs.
cl004.cl.arc.internal: mmremote: This node does not belong to a GPFS
cluster.
mmdsh: cl004.cl.
Maybe multipath is not seeing all of the wwns?
multipath -v3 | grep ^51855 look ok?
For some unknown reason multipath does not see our sandisk array...we have
to add them to the end of /etc/multipath/wwids file
On Fri, Jan 20, 2017 at 10:32 AM, David D. Johnson
wrote:
> We have most of our GP
I have messages like these frequent my logs:
Tue Jan 17 11:25:49.731 2017: [E] VERBS RDMA rdma write error
IBV_WC_REM_ACCESS_ERR to 10.51.10.5 (cl005) on mlx5_0 port 1 fabnum 0
vendor_err 136
Tue Jan 17 11:25:49.732 2017: [E] VERBS RDMA closed connection to
10.51.10.5 (cl005) on mlx5_0 port 1 fabnu
Our intent was to have ccr turned off since all nodes are quorum in the
server cluster:
Considering this:
[root@cl001 ~]# mmfsadm dump config | grep -i ccr
! ccrEnabled 0
ccrMaxChallengeCheckRetries 4
ccr : 0 (cluster configuration repository)
ccr : 1 (cluster c
> Fax: +972-3-916-5672 <+972%203-916-5672>
> Mobile: +972-52-8395593 <+972%2052-839-5593>
> e-mail: y...@il.ibm.com
> *IBM Israel* <http://www.ibm.com/il/he/>
>
>
>
>
>
>
>
> From:"J. Eric Wonderley"
> To:gpfsug main di
Hi Yaron:
This is the filesystem:
[root@cl005 net]# mmlsdisk work
disk driver sector failure holds
holdsstorage
name type size group metadata data status
availability pool
-- --- - --
..@spectrumscale.org [mailto:gpfsug-discuss-
> boun...@spectrumscale.org] *On Behalf Of *J. Eric Wonderley
> *Sent:* Thursday, January 05, 2017 2:01 PM
> *To:* gpfsug main discussion list
> *Subject:* [gpfsug-discuss] nsd not adding with one quorum node down?
>
>
>
> I have one quorum node do
I have one quorum node down and attempting to add a nsd to a fs:
[root@cl005 ~]# mmadddisk home -F add_1_flh_home -v no |& tee
/root/adddisk_flh_home.out
Verifying file system configuration information ...
The following disks of home will be formatted on node cl003:
r10f1e5: size 1879610 MB
Ex
[root@cl001 ~]# cat chnsd_home_flh
%nsd: nsd=r10f1e5 servers=cl008,cl001,cl002,cl003,cl004,cl005,cl006,cl007
%nsd: nsd=r10f6e5 servers=cl007,cl008,cl001,cl002,cl003,cl004,cl005,cl006
%nsd: nsd=r10f1e6 servers=cl006,cl007,cl008,cl001,cl002,cl003,cl004,cl005
%nsd: nsd=r10f6e6 servers=cl005,cl006,cl00
Hi Michael:
I was about to ask a similar question about nested filesets.
I have this setup:
[root@cl001 ~]# mmlsfileset home
Filesets in file system 'home':
Name StatusPath
root Linked/gpfs/home
groupLinked/gpfs/home/group
pr
disks were removed.
> Can do multiple that why and us the entire cluster to move data if you
> want.
>
> On 12/1/16 1:10 PM, J. Eric Wonderley wrote:
>
> I have a few misconfigured disk groups and I have a few same size
> correctly configured disk groups.
>
> Is there an
I have a few misconfigured disk groups and I have a few same size correctly
configured disk groups.
Is there any (dis)advantage to running mmrpldisk over mmdeldisk and
mmadddisk? Everytime I have ever run mmdeldisk...it been somewhat
painful(even with qos) process.
___
ot use the size
>>> of the file to determine the placement of the file in a GPFS Storage Pool.
>>> This is because GPFS has no idea what the file size will be when the file
>>> is open()’d for writing.
>>>
>>>
>>>
>>> Hope that helps!
>
I wanted to do something like this...
[root@cl001 ~]# cat /opt/gpfs/home.ply
/*Failsafe migration of old small files back to spinning media pool(fc_8T)
*/
RULE 'theshold' MIGRATE FROM POOL 'system' THRESHOLD(90,70)
WEIGHT(ACCESS_TIME) TO POOL 'fc_8T'
/*Write files larger than 16MB to pool called "
We have the need to move data from one set of spindles to another.
Are there any performance or availability considerations when choosing to
do either a migration policy or a restripe to make this move? I did
discover that a restripe only works within the same pool...even though you
setup two poo
It defaults to 4k:
mmlsfs testbs8M -i
flagvaluedescription
---
---
-i 4096 Inode size in bytes
I think you can make as small as 512b. Gpfs will store
42 matches
Mail list logo
