[Gluster-users] Permission denied
Hello, I'm getting Permission Denied error when I connect to brick server. I'm using 2(distributed)*2(replication) configuration. Apache runs with apache user/group and my apache module invoke GlusterFSClient. Server daemon and Client libraries were installed through CentOS rpms. Here're client log. I tried with fuse client with same configuration file from root account. All File operations were fine. Is there any caution when I run non-root privileges ? [2010-04-25 21:21:58] D [xlator.c:740:xlator_set_type] xlator: dlsym(notify) on /usr/lib64/glusterfs/3.0.4/xlator/performance/io-threads.so: undefined symbol: notify -- neglecting [2010-04-25 21:21:58] D [xlator.c:745:xlator_set_type] xlator: dlsym(dumpops) on /usr/lib64/glusterfs/3.0.4/xlator/performance/io-threads.so: undefine d symbol: dumpops -- neglecting [2010-04-25 21:21:58] D [xlator.c:740:xlator_set_type] xlator: dlsym(notify) on /usr/lib64/glusterfs/3.0.4/xlator/performance/read-ahead.so: undefined symbol: notify -- neglecting [2010-04-25 21:21:58] D [xlator.c:740:xlator_set_type] xlator: dlsym(notify) on /usr/lib64/glusterfs/3.0.4/xlator/performance/write-behind.so: undefin ed symbol: notify -- neglecting [2010-04-25 21:21:58] D [xlator.c:740:xlator_set_type] xlator: dlsym(notify) on /usr/lib64/glusterfs/3.0.4/xlator/performance/io-cache.so: undefined s ymbol: notify -- neglecting [2010-04-25 21:21:58] D [xlator.c:745:xlator_set_type] xlator: dlsym(dumpops) on /usr/lib64/glusterfs/3.0.4/xlator/performance/io-cache.so: undefined symbol: dumpops -- neglecting [2010-04-25 21:21:58] D [xlator.c:740:xlator_set_type] xlator: dlsym(notify) on /usr/lib64/glusterfs/3.0.4/xlator/performance/quick-read.so: undefined symbol: notify -- neglecting [2010-04-25 21:21:58] D [write-behind.c:2526:init] writebehind: disabling write-behind for first 1 bytes [2010-04-25 21:21:58] D [read-ahead.c:884:init] readahead: Using conf-page_count = 16 [2010-04-25 21:21:58] D [io-threads.c:2841:init] iothreads: io-threads: Autoscaling: off, min_threads: 32, max_threads: 32 [2010-04-25 21:21:58] D [client-protocol.c:6603:init] client04: defaulting frame-timeout to 30mins [2010-04-25 21:21:58] D [client-protocol.c:6614:init] client04: defaulting ping-timeout to 42 [2010-04-25 21:21:58] D [transport.c:123:transport_load] transport: attempt to load file /usr/lib64/glusterfs/3.0.4/transport/socket.so [2010-04-25 21:21:58] D [xlator.c:285:_volume_option_value_validate] client04: no range check required for 'option remote-port 6996' [2010-04-25 21:21:58] D [transport.c:123:transport_load] transport: attempt to load file /usr/lib64/glusterfs/3.0.4/transport/socket.so [2010-04-25 21:21:58] D [xlator.c:285:_volume_option_value_validate] client04: no range check required for 'option remote-port 6996' [2010-04-25 21:21:58] W [client-protocol.c:6664:init] client04: WARNING: Failed to set 'ulimit -n 1M': Operation not permitted [2010-04-25 21:21:58] D [client-protocol.c:6672:init] client04: Failed to set max open fd to 64k: Operation not permitted [2010-04-25 21:21:58] D [client-protocol.c:6603:init] client03: defaulting frame-timeout to 30mins [2010-04-25 21:21:58] D [client-protocol.c:6614:init] client03: defaulting ping-timeout to 42 [2010-04-25 21:21:58] D [transport.c:123:transport_load] transport: attempt to load file /usr/lib64/glusterfs/3.0.4/transport/socket.so [2010-04-25 21:21:58] D [xlator.c:285:_volume_option_value_validate] client03: no range check required for 'option remote-port 6996' [2010-04-25 21:21:58] D [transport.c:123:transport_load] transport: attempt to load file /usr/lib64/glusterfs/3.0.4/transport/socket.so [2010-04-25 21:21:58] D [xlator.c:285:_volume_option_value_validate] client03: no range check required for 'option remote-port 6996' [2010-04-25 21:21:58] W [client-protocol.c:6664:init] client03: WARNING: Failed to set 'ulimit -n 1M': Operation not permitted [2010-04-25 21:21:58] D [client-protocol.c:6672:init] client03: Failed to set max open fd to 64k: Operation not permitted [2010-04-25 21:23:09] E [socket.c:1089:socket_connect] client04: connection attempt failed (Permission denied) [2010-04-25 21:23:09] E [afr.c:2662:notify] replicate02: All subvolumes are down. Going offline until atleast one of them comes back up. [2010-04-25 21:23:09] W [name.c:443:client_bind] client04: cannot bind inet socket (34) to port less than 1024 (Permission denied) [2010-04-25 21:23:09] E [socket.c:1089:socket_connect] client04: connection attempt failed (Permission denied) [2010-04-25 21:23:09] E [afr.c:2662:notify] replicate02: All subvolumes are down. Going offline until atleast one of them comes back up. [2010-04-25 21:23:11] W [name.c:443:client_bind] client01: cannot bind inet socket (35) to port less than 1024 (Permission denied) [2010-04-25 21:23:11] E [socket.c:1089:socket_connect] client01: connection attempt failed (Permission denied) [2010-04-25 21:23:11] E [afr.c:2662:notify] replicate01: All subvolumes are down. Going
Re: [Gluster-users] New 3.0.4 install, missing something on client config
Changed all instances of brick to brick1 in client config... same error. level56 glusterfs # glusterfs --debug -f ./glusterfs.vol /data [2010-04-26 06:24:09] D [glusterfsd.c:432:_get_specfp] glusterfs: loading volume file ./glusterfs.vol [2010-04-26 06:24:09] D [xlator.c:740:xlator_set_type] xlator: dlsym(notify) on /usr/local/lib/glusterfs/3.0.4/xlator/performance/write-behind.so: undefined symbol: notify -- neglecting [2010-04-26 06:24:09] D [xlator.c:740:xlator_set_type] xlator: dlsym(notify) on /usr/local/lib/glusterfs/3.0.4/xlator/performance/io-cache.so: undefined symbol: notify -- neglecting [2010-04-26 06:24:09] D [xlator.c:745:xlator_set_type] xlator: dlsym(dumpops) on /usr/local/lib/glusterfs/3.0.4/xlator/performance/io-cache.so: undefined symbol: dumpops -- neglecting Version : glusterfs 3.0.4 built on Apr 25 2010 07:12:27 git: v3.0.4 Starting Time: 2010-04-26 06:24:09 Command line : glusterfs --debug -f ./glusterfs.vol /data PID : 26993 System name : Linux Nodename : level56.lite3d.com Kernel Release : 2.6.26-gentoo-r3 Hardware Identifier: x86_64 Given volfile: +--+ 1: volume level56 2: type protocol/client 3: option transport-type tcp 4: option remote-host 10.1.85.36 5: option transport.socket.nodelay on 6: option remote-subvolume brick1 7: end-volume 8: 9: volume level57 10: type protocol/client 11: option transport-type tcp 12: option remote-host 10.1.85.37 13: option transport.socket.nodelay on 14: option remote-subvolume brick1 15: end-volume 16: 17: volume level58 18: type protocol/client 19: option transport-type tcp 20: option remote-host 10.1.85.38 21: option transport.socket.nodelay on 22: option remote-subvolume brick1 23: end-volume 24: 25: volume level59 26: type protocol/client 27: option transport-type tcp 28: option remote-host 10.1.85.39 29: option transport.socket.nodelay on 30: option remote-subvolume brick1 31: end-volume 32: 33: volume level60 34: type protocol/client 35: option transport-type tcp 36: option remote-host 10.1.85.40 37: option transport.socket.nodelay on 38: option remote-subvolume brick1 39: end-volume 40: 41: #61 is not here yet... 42: #volume level61 43: # type protocol/client 44: # option transport-type tcp 45: # option remote-host 10.1.85.41 46: # option transport.socket.nodelay on 47: # option remote-subvolume brick1 48: #end-volume 49: 50: volume level62 51: type protocol/client 52: option transport-type tcp 53: option remote-host 10.1.85.42 54: option transport.socket.nodelay on 55: option remote-subvolume brick1 56: end-volume 57: 58: volume level63 59: type protocol/client 60: option transport-type tcp 61: option remote-host 10.1.85.43 62: option transport.socket.nodelay on 63: option remote-subvolume brick1 64: end-volume 65: 66: volume level64 67: type protocol/client 68: option transport-type tcp 69: option remote-host 10.1.85.44 70: option transport.socket.nodelay on 71: option remote-subvolume brick1 72: end-volume 73: 74: volume level65 75: type protocol/client 76: option transport-type tcp 77: option remote-host 10.1.85.45 78: option transport.socket.nodelay on 79: option remote-subvolume brick1 80: end-volume 81: 82: volume level66 83: type protocol/client 84: option transport-type tcp 85: option remote-host 10.1.85.46 86: option transport.socket.nodelay on 87: option remote-subvolume brick1 88: end-volume 89: 90: volume level67 91: type protocol/client 92: option transport-type tcp 93: option remote-host 10.1.85.47 94: option transport.socket.nodelay on 95: option remote-subvolume brick1 96: end-volume 97: 98: volume level68 99: type protocol/client 100: option transport-type tcp 101: option remote-host 10.1.85.48 102: option transport.socket.nodelay on 103: option remote-subvolume brick1 104: end-volume 105: 106: volume level69 107: type protocol/client 108: option transport-type tcp 109: option remote-host 10.1.85.49 110: option transport.socket.nodelay on 111: option remote-subvolume brick1 112: end-volume 113: 114: 115: volume distribute 116: type cluster/distribute 117: option lookup-unhashed off 118: subvolumes level56 level57 level58 level59 level60 level62 level63 level64 level65 level66 level67 level68 level69 119: end-volume 120: 121: volume wb 122: type performance/write-behind 123: option cache-size 5MB# default is equal to aggregate-size 124: option flush-behind off# default is 'off' 125: subvolumes distribute 126: end-volume 127: 128: volume ioc 129: type performance/io-cache 130: option cache-size 128MB 131: subvolumes wb 132: end-volume
Re: [Gluster-users] Transport endpoint not connected
Here is the relevant crash section: patchset: v3.0.4 signal received: 11 time of crash: 2010-04-23 21:40:40 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.0.4 /lib/libc.so.6[0x7ffd0d809100] /usr/local/lib/glusterfs/3.0.4/xlator/performance/read-ahead.so(ra_fstat +0x82)[0 x7ffd0c968d22] /usr/local/lib/libglusterfs.so.0(default_fstat+0xcb)[0x7ffd0df7411b] /usr/local/lib/glusterfs/3.0.4/xlator/performance/quick-read.so(qr_fstat +0x113)[ 0x7ffd0c5570a3] /usr/local/lib/glusterfs/3.0.4/xlator/performance/write-behind.so(wb_fst at_helpe r+0xcb)[0x7ffd0c346adb] /usr/local/lib/libglusterfs.so.0(call_resume+0x390)[0x7ffd0df7cf60] /usr/local/lib/glusterfs/3.0.4/xlator/performance/write-behind.so(wb_res ume_othe r_requests+0x58)[0x7ffd0c349938] /usr/local/lib/glusterfs/3.0.4/xlator/performance/write-behind.so(wb_pro cess_que ue+0xe1)[0x7ffd0c348251] /usr/local/lib/glusterfs/3.0.4/xlator/performance/write-behind.so(wb_fst at+0x20a )[0x7ffd0c34a87a] /usr/local/lib/libglusterfs.so.0(default_fstat+0xcb)[0x7ffd0df7411b] /usr/local/lib/glusterfs/3.0.4/xlator/mount/fuse.so[0x7ffd0bf23a36] /usr/local/lib/glusterfs/3.0.4/xlator/mount/fuse.so[0x7ffd0bf246b6] /lib/libpthread.so.0[0x7ffd0db3f3f7] /lib/libc.so.6(clone+0x6d)[0x7ffd0d8aeb4d] And Startup section: - Version : glusterfs 3.0.4 built on Apr 19 2010 16:37:50 git: v3.0.4 Starting Time: 2010-04-26 10:00:59 Command line : /usr/local/sbin/glusterfs --log-level=NORMAL --volfile=/etc/glust erfs/repstore1-tcp.vol /data/import PID : 5910 System name : Linux Nodename : w2 Kernel Release : 2.6.24-27-server Hardware Identifier: x86_64 Given volfile: +--- ---+ 1: ## file auto generated by /usr/local/bin/glusterfs-volgen (mount.vol) 2: # Cmd line: 3: # $ /usr/local/bin/glusterfs-volgen --name repstore1 --raid 1 10.10.130.11:/data/export 10.10.130.12:/data/export 4: 5: # RAID 1 6: # TRANSPORT-TYPE tcp 7: volume 10.10.130.12-1 8: type protocol/client 9: option transport-type tcp 10: option remote-host 10.10.130.12 11: option transport.socket.nodelay on 12: option transport.remote-port 6996 13: option remote-subvolume brick1 14: end-volume 15: 16: volume 10.10.130.11-1 17: type protocol/client 18: option transport-type tcp 19: option remote-host 10.10.130.11 20: option transport.socket.nodelay on 21: option transport.remote-port 6996 22: option remote-subvolume brick1 23: end-volume 24: 25: volume mirror-0 26: type cluster/replicate 27: subvolumes 10.10.130.11-1 10.10.130.12-1 28: end-volume 29: 30: volume readahead 31: type performance/read-ahead 32: option page-count 4 33: subvolumes mirror-0 34: end-volume 35: 36: volume iocache 37: type performance/io-cache 38: option cache-size `echo $(( $(grep 'MemTotal' /proc/meminfo | sed 's/[^0-9]//g') / 5120 ))`MB 39: option cache-timeout 1 40: subvolumes readahead 41: end-volume 42: 43: volume quickread 44: type performance/quick-read 45: option cache-timeout 1 46: option max-file-size 64kB 47: subvolumes iocache 48: end-volume 49: 50: volume writebehind 51: type performance/write-behind 52: option cache-size 4MB 53: subvolumes quickread 54: end-volume 55: 56: volume statprefetch 57: type performance/stat-prefetch 58: subvolumes writebehind 59: end-volume 60: -Original Message- From: Vijay Bellur [mailto:vi...@gluster.com] Sent: 22 April 2010 18:40 To: Joe Warren-Meeks Cc: gluster-users@gluster.org Subject: Re: [Gluster-users] Transport endpoint not connected Hi Joe, Can you please share the complete client log file? Thanks, Vijay Joe Warren-Meeks wrote: Hey guys, I've recently implemented gluster to share webcontent read-write between two servers. Version : glusterfs 3.0.4 built on Apr 19 2010 16:37:50 Fuse: 2.7.2-1ubuntu2.1 Platform: ubuntu 8.04LTS I used the following command to generate my configs: /usr/local/bin/glusterfs-volgen --name repstore1 --raid 1 10.10.130.11:/data/export 10.10.130.12:/data/export And mount them on each of the servers as so: /etc/fstab: /etc/glusterfs/repstore1-tcp.vol /data/import glusterfs defaults 0 0 Every 12 hours or so, one or other of the servers will lose the mount and error with: df: `/data/import': Transport endpoint is not connected And I get the following in my logfile: patchset: v3.0.4 signal received: 11 time of crash: 2010-04-22 11:41:10 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1
