[Gluster-users] Heal failed entries glsuter 3.6.1
Hi, I have a question about heal process. Sometimes appear heal entries failed after execute gluster volume heal Volname statistics. When can happen these cases? Thanks ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Upgrade gluster 3.6.1 to 3.7.1
Hi, We want upgrade gluster 3.6.1 to 3.7.1. I have a question about quota, must take into account the change in configuration? In this case, what steps should follow? Thanks. ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Monitorig gluster 3.6.1
Hi, I have monitoring gluster with scripts that lunch scripts. All scripts are redirected to a one script that check if is active any process glusterd and if the repsonse its false, the script lunch the check. All checks are: - gluster volume volname info - gluster volume heal volname info - gluster volume heal volname split-brain - gluster volume volname status detail - gluster volume volname statistics Since I enable the monitoring in our pre-production gluster, the gluster is down 2 times. We suspect that the monitoring are overloading but should not. The question is, there any way to check those states otherwise? Thanks ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Gluster 3.6.1
Hi, I have a cluster with 3 nodes on pre-production. Yesterday, one node was down. The errror that I have seen is that: [2015-05-28 19:04:27.305560] E [glusterd-syncop.c:1578:gd_sync_task_begin] 0-management: Unable to acquire lock for cfe-gv1 The message "I [MSGID: 106006] [glusterd-handler.c:4257:__glusterd_nodesvc_rpc_notify] 0-management: nfs has disconnected from glusterd." repeated 5 times between [2015-05-28 19:04:09.346088] and [2015-05-28 19:04:24.349191] pending frames: frame : type(0) op(0) patchset: git://git.gluster.com/glusterfs.git signal received: 11 time of crash: 2015-05-28 19:04:27 configuration details: argp 1 backtrace 1 dlfcn 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.6.1 /usr/lib64/libglusterfs.so.0(_gf_msg_backtrace_nomem+0xb2)[0x7fd86e2f1232] /usr/lib64/libglusterfs.so.0(gf_print_trace+0x32d)[0x7fd86e30871d] /usr/lib64/libc.so.6(+0x35640)[0x7fd86d30c640] /usr/lib64/glusterfs/3.6.1/xlator/mgmt/glusterd.so(glusterd_remove_pending_entry+0x2c)[0x7fd85f52450c] /usr/lib64/glusterfs/3.6.1/xlator/mgmt/glusterd.so(+0x5ae28)[0x7fd85f511e28] /usr/lib64/glusterfs/3.6.1/xlator/mgmt/glusterd.so(glusterd_op_sm+0x237)[0x7fd85f50f027] /usr/lib64/glusterfs/3.6.1/xlator/mgmt/glusterd.so(__glusterd_brick_op_cbk+0x2fe)[0x7fd85f53be5e] /usr/lib64/glusterfs/3.6.1/xlator/mgmt/glusterd.so(glusterd_big_locked_cbk+0x4c)[0x7fd85f53d48c] /usr/lib64/libgfrpc.so.0(rpc_clnt_handle_reply+0x90)[0x7fd86e0c50b0] /usr/lib64/libgfrpc.so.0(rpc_clnt_notify+0x171)[0x7fd86e0c5321] /usr/lib64/libgfrpc.so.0(rpc_transport_notify+0x23)[0x7fd86e0c1273] /usr/lib64/glusterfs/3.6.1/rpc-transport/socket.so(+0x8530)[0x7fd85d17d530] /usr/lib64/glusterfs/3.6.1/rpc-transport/socket.so(+0xace4)[0x7fd85d17fce4] /usr/lib64/libglusterfs.so.0(+0x76322)[0x7fd86e346322] /usr/sbin/glusterd(main+0x502)[0x7fd86e79afb2] /usr/lib64/libc.so.6(__libc_start_main+0xf5)[0x7fd86d2f8af5] /usr/sbin/glusterd(+0x6351)[0x7fd86e79b351] - That is a problem with software? is a bug ? Thanks. ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Upgrade Glsuter 3.7 : Issue with quota
Hi, I have upgraded gluster from release 6.3, with quota enable, to release 3.7, and when I have restarted the nodes, the log lunch a error: E [MSGID: 106012] [glusterd-utils.c:2670:glusterd_compare_friend_volume] 0-management: Cksums of quota configuration of volume cfe-gv1 differ. local cksum = 2736284911, remote cksum = 4092896828 on peer srv-vln-des3 Where is the problem? Thanks. ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Metadata filesystem XFS gluster 3.6
Hi,
I had a problem with a filesystem xfs on gluster. The filesystem metadata
was filled:
Mar 27 13:58:18 srv-vln-des2 kernel: device-mapper: space map metadata:
unable to allocate new metadata block
Mar 27 13:58:18 srv-vln-des2 kernel: device-mapper: thin: 252:2: metadata
operation 'dm_thin_insert_block' failed: error = -28
Mar 27 13:58:18 srv-vln-des2 kernel: device-mapper: thin: 252:2: aborting
current metadata transaction
Mar 27 13:58:18 srv-vln-des2 kernel: device-mapper: thin: 252:2: switching
pool to read-only mode
Mar 27 13:58:18 srv-vln-des2 kernel: XFS (dm-4): metadata I/O error: block
0x701830 ("xfs_buf_iodone_callbacks") error 5 numblks 8
Mar 27 13:58:18 srv-vln-des2 kernel: attempt to access beyond end of device
Mar 27 13:58:18 srv-vln-des2 kernel: dm-0: rw=0, want=562056, limit=24576
Mar 27 13:58:18 srv-vln-des2 kernel: device-mapper: thin:
process_bio_read_only: dm_thin_find_block() failed: error = -5
Mar 27 13:58:18 srv-vln-des2 kernel: attempt to access beyond end of device
Mar 27 13:58:18 srv-vln-des2 kernel: dm-0: rw=0, want=562056, limit=24576
Mar 27 13:58:18 srv-vln-des2 kernel: device-mapper: thin:
process_bio_read_only: dm_thin_find_block() failed: error = -5
Mar 27 13:58:18 srv-vln-des2 kernel: XFS (dm-4): metadata I/O error: block
0x68047c ("xlog_iodone") error 5 numblks 64
Mar 27 13:58:18 srv-vln-des2 kernel: XFS (dm-4): xfs_do_force_shutdown(0x2)
called from line 1170 of file fs/xfs/xfs_log.c. Return address =
0xa012a4c1
Mar 27 13:58:18 srv-vln-des2 kernel: XFS (dm-4): Log I/O Error Detected.
Shutting down filesystem
Mar 27 13:58:18 srv-vln-des2 kernel: XFS (dm-4): Please umount the
filesystem and rectify the problem(s)
Mar 27 13:58:18 srv-vln-des2 kernel: attempt to access beyond end of device
Mar 27 13:58:18 srv-vln-des2 kernel: dm-0: rw=0, want=562056, limit=24576
Mar 27 13:58:18 srv-vln-des2 kernel: device-mapper: thin:
process_bio_read_only: dm_thin_find_block() failed: error = -5
Mar 27 13:58:18 srv-vln-des2 kernel: XFS (dm-4): metadata I/O error: block
0x6804bc ("xlog_iodone") error 5 numblks 64
Mar 27 13:58:18 srv-vln-des2 kernel: XFS (dm-4): xfs_do_force_shutdown(0x2)
called from line 1170 of file fs/xfs/xfs_log.c. Return address =
0xa012a4c1
Mar 27 13:58:18 srv-vln-des2 kernel: attempt to access beyond end of device
Mar 27 13:58:18 srv-vln-des2 kernel: XFS (dm-4): xfs_log_force: error 5
returned.
Mar 27 13:58:18 srv-vln-des2 kernel: dm-0: rw=0, want=562056, limit=24576
Mar 27 13:58:18 srv-vln-des2 kernel: device-mapper: thin:
process_bio_read_only: dm_thin_find_block() failed: error = -5
Mar 27 13:58:18 srv-vln-des2 kernel: attempt to access beyond end of device
Mar 27 13:58:18 srv-vln-des2 kernel: dm-0: rw=0, want=562056, limit=24576
Mar 27 13:58:18 srv-vln-des2 kernel: device-mapper: thin:
process_bio_read_only: dm_thin_find_block() failed: error = -5
Mar 27 13:58:18 srv-vln-des2 kernel: attempt to access beyond end of device
Mar 27 13:58:18 srv-vln-des2 kernel: dm-0: rw=0, want=562056, limit=24576
Mar 27 13:58:18 srv-vln-des2 kernel: device-mapper: thin:
process_bio_read_only: dm_thin_find_block() failed: error = -5
Mar 27 13:58:18 srv-vln-des2 kernel: XFS (dm-4): metadata I/O error: block
0x6804fc ("xlog_iodone") error 5 numblks 64
Mar 27 13:58:18 srv-vln-des2 kernel: XFS (dm-4): xfs_do_force_shutdown(0x2)
called from line 1170 of file fs/xfs/xfs_log.c. Return address =
0xa012a4c
After that, gluster was shutdown and with it the 2 server are shtudown too.
The lvm partition was missing and so far I haven't been able restore the
file system. All data is missing??
I don't understand the situation and I don't know if it's due a xfs
filesystem or glusterfs fail. Someone it has been this situation?
Thanks.
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] "Out of memory" Gluster 3.6.2
Hi, Today, Glusterd daemon has been killed due to excessive memory consumption: [3505254.762715] Out of memory: Kill process 7780 (glusterd) score 581 or sacrifice child [3505254.763451] Killed process 7780 (glusterd) total-vm:3537640kB, anon-rss:1205240kB, file-rss:672kB I have installed gluster 3.6.2 on centos 7. There in any way to avoid this whithout the need to kill the process?Simply need more memory? Thanks. ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] Issue glusterfs 3.6.2
Hi, Sorry, I have not explained well. I don't mean that I have lunch this comand simultaneously, I am using zabbix and I have defined diferent time intervals for this but comes a point at wich I can't luch anythig command over the cluster, such as "gluster volume status". Is there any way around this? Thanks 2015-03-18 11:36 GMT+01:00 Atin Mukherjee : > > > On 03/18/2015 03:24 PM, Félix de Lelelis wrote: > > Hi, > > > > I have a problem with glusterfs 3.6. I am monitoring it with scripts that > > lunch "gluster volume status VONAME detail" and "gluster volume profile > > VOLNAME info". When this scripts are running about 1-2 hours, with check > > every 1 minute, gluster is blocked and the node generates "Another > > transaction is in progress". The only way to restore the situation is > > restart glusterd. > > > > Can you help me please. > This is expected. Concurrent transactions on a same volumes are not > allowed. There is a cluster wide volume lock maintained by glusterd. In > this case since both these commands are for VOLNAME, any one of two > would throw an error saying "Another transaction is in progress". If you > ensure that there is a time gap between these two commands in your > script, you can get rid of this situation. > > ~Atin > > > > > > Thanks > > > > > > > > ___ > > Gluster-users mailing list > > Gluster-users@gluster.org > > http://www.gluster.org/mailman/listinfo/gluster-users > > > > -- > ~Atin > ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Issue glusterfs 3.6.2
Hi, I have a problem with glusterfs 3.6. I am monitoring it with scripts that lunch "gluster volume status VONAME detail" and "gluster volume profile VOLNAME info". When this scripts are running about 1-2 hours, with check every 1 minute, gluster is blocked and the node generates "Another transaction is in progress". The only way to restore the situation is restart glusterd. Can you help me please. Thanks ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Gluster 3.6 issue
Hi, I have a cluster with 2 nodes, and sometimes when I lunch gluster volume status, appear a error on the log: [2015-03-16 17:24:25.215352] E [glusterd-utils.c:7364:glusterd_add_inode_size_to_dict] 0-management: xfs_info exited with non-zero exit status [2015-03-16 17:24:25.215379] E [glusterd-utils.c:7390:glusterd_add_inode_size_to_dict] 0-management: failed to get inode size That you can be due? Thanks ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] FOPS gluster 3.6
Hi, Which is the difference between these types of fops: read, readdir and readdirp. I am monitoring and I am interesting on read and write operations but when lunch cat comand over the client only is modified readdirp on the profile. Thanks. ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Monitoring gluster 3.6
Hi, Someone know how obtain stats over fops or I/O operations in gluster? The idea is integrate this scripts with zabbix. Thanks. ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Geo-replication session
Hi, I am testing geo-replication in gluster-3.6. I have created 2 sessions but when I deleted the last one glusterd yet has register it: [2015-02-23 08:51:25.440521] I [glusterd-geo-rep.c:3907:glusterd_get_gsync_status_mst_slv] 0-: geo-replication status prueba srv-vln-des3-priv1::back-cfe :session is not active [2015-02-23 08:51:25.440584] W [glusterd-geo-rep.c:1840:glusterd_get_statefile_name] 0-: Config file (/var/lib/glusterd/geo-replication/prueba_srv-vln-des3-priv1_back-cfe/gsyncd.conf) missing. Looking for template config file (/var/lib/glusterd/geo-replication/gsyncd_template.conf) [2015-02-23 08:51:25.440598] I [glusterd-geo-rep.c:1849:glusterd_get_statefile_name] 0-: Using default config template(/var/lib/glusterd/geo-replication/gsyncd_template.conf). [2015-02-23 08:51:25.524719] I [glusterd-geo-rep.c:3926:glusterd_get_gsync_status_mst_slv] 0-: /var/lib/glusterd/geo-replication/prueba_srv-vln-des3-priv1_back-cfe/ssh%3A%2F%2Froot%40192.168.64.207%3Agluster%3A%2F%2F127.0.0.1%3Aback-cfe.status statefile not present. [2015-02-23 08:51:25.524880] I [glusterd-handler.c:1280:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2015-02-23 08:51:25.627568] I [glusterd-handler.c:3803:__glusterd_handle_status_volume] 0-management: Received status volume req for volume prueba [2015-02-23 08:51:25.627948] W [glusterd-locks.c:550:glusterd_mgmt_v3_lock] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn+0x186)[0x7fecabad94c6] (--> /usr/lib64/glusterfs/3.6.2/xlator/mgmt/glusterd.so(glusterd_mgmt_v3_lock+0x4cb)[0x7feca157510b] (--> /usr/lib64/glusterfs/3.6.2/xlator/mgmt/glusterd.so(gd_sync_task_begin+0x57d)[0x7feca15726bd] (--> /usr/lib64/glusterfs/3.6.2/xlator/mgmt/glusterd.so(glusterd_op_begin_synctask+0x2c)[0x7feca15729ec] (--> /usr/lib64/glusterfs/3.6.2/xlator/mgmt/glusterd.so(__glusterd_handle_status_volume+0x152)[0x7feca14d9852] ) 0-management: Lock for prueba held by af083e0a-737e-4aa0-9f00-0e0efcd477df [2015-02-23 08:51:25.628044] E [glusterd-syncop.c:1578:gd_sync_task_begin] 0-management: Unable to acquire lock for prueba How I can reset this situation? Thanks ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Changelogs on gluster 3.6
Hi, There is anyway to take information about the last changelog that is applied on slave and master node in geo-replication? Thanks ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Split-brain info gluster 3.6
Hi, I generated a split-brain condition, and I solved it but with "gluster volume heal vol_name info split-brain" yet I can see past entries solved: Number of entries: 3 atpath on brick --- 2015-02-19 17:13:08 /split 2015-02-19 17:14:09 /split 2015-02-19 17:15:10 /split Brick srv-vln-des2-priv1:/gfs-to-snap/prueba/brick1/brick Number of entries: 4 atpath on brick --- 2015-02-19 17:09:32 /split 2015-02-19 17:13:08 /split 2015-02-19 17:14:09 /split 2015-02-19 17:15:10 /split How can I reset that entries? Thanks ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Splitmount on gluster 3.6
Hi, I wolud like to know if splitmount utility is useful for gluster version upper 3.3. I think that since 3.3 version I need to delete the hardlink also in split-brain case, it's true? Thank's ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Simulate split-brain on gluster 3.6
Hi, I am simulating a split brain condition on my cluster but I don't be able it. I have disconnected the nodes and creating a file with the same name and different contents but always the self-heal process take the last copy of the file. How can create thos condition? thanks ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Warning: "W [socket.c:611:__socket_rwv] 0-management: readv on "
Hi, The last week upgraded us cluster to 3.6 version. I noticed in the log the following error: W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/f3fcde54ca5d30115274155a37baa079.socket failed (Invalid argument) It is due a nfs daemon? Thanks. ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Changelog dir in gluster 3.6
Hi, I don't understand if the changelog is needed to replica and geo-replica situations, so the filesystem is fill of a lot of that files in /.glusterfs/changelogs. Is there anyway to reduce or waive directory? Thanks. ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] upgrade to gluster 3.6
Hi, I don't know if this way it's the correct, so if I reported to other account, please let me know. I have a problem after upgrade my cluster to version 3.6 from version 3.5. When I have started the volumes, I have seen a lot of errors: [2015-02-11 11:23:18.231142] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/032613d904825273481612e900965093.socket failed (Invalid argument) [2015-02-11 11:23:21.231749] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/032613d904825273481612e900965093.socket failed (Invalid argument) [2015-02-11 11:23:24.232092] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/032613d904825273481612e900965093.socket failed (Invalid argument) [2015-02-11 11:23:27.232648] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/032613d904825273481612e900965093.socket failed (Invalid argument) [2015-02-11 11:23:30.233272] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/032613d904825273481612e900965093.socket failed (Invalid argument) [2015-02-11 11:23:33.233672] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/032613d904825273481612e900965093.socket failed (Invalid argument) [2015-02-11 11:23:36.234255] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/032613d904825273481612e900965093.socket failed (Invalid argument) [2015-02-11 11:23:39.234688] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/032613d904825273481612e900965093.socket failed (Invalid argument) [2015-02-11 11:23:42.235269] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/032613d904825273481612e900965093.socket failed (Invalid argument) [2015-02-11 11:23:45.235867] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/032613d904825273481612e900965093.socket failed (Invalid argument) [2015-02-11 11:23:48.236499] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/032613d904825273481612e900965093.socket failed (Invalid argument) [2015-02-11 11:23:51.237021] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/032613d904825273481612e900965093.socket failed (Invalid argument) Can you help me please? Thanks. ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
