Hi, fixed. /case closed
I write monday a article about. - alex On Fri, Dec 9, 2011 at 9:40 AM, alo alt <wget.n...@googlemail.com> wrote: > Hi folks, > > I research at the moment a connection over fuse and samba for > hadoop-cluster. Its my private playground, but I have some issues I can't > figure out why they happen. > > I use RHEL5.7, packetlist: > rpm -qa|grep samba > samba-3.0.33-3.29.el5_7.4.x86_64 > samba-common-3.0.33-3.29.el5_7.4.x86_64 > > 2 servers, one provides samba-shares and one I use as client. > > Samba-Server (HOST2): > FuseFS: > hadoop-fuse-dfs#dfs://HOSTNAME:9000 /123/hdfs fuse usetrash,rw 0 0 > > mount: > fuse on /123/hdfs type fuse > (rw,nosuid,nodev,allow_other,default_permissions) > > the mount /123 is accessible over ssh, cp, rsnyc from any host. Now I was > using NFS for populating, but the performance was not as I expected. So I > switched to samba and get real good results for reading. > > Cifs-Client(HOST1): > mount -t cifs -o username=BLUBB,domain=hdfs,rw,uid=104 //lHOST2/hdfs /test > > config: > > [global] > workgroup = MYGROUP > server string = Samba Server Version %v > socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192 > security = user > smb passwd file = /etc/samba/smbpasswd > #passdb backend = tdbsam > > > [hdfs] > comment = hhh > path = /123 > public = yes > writable = yes > printable = no > create mask = 0744 > force user = hdfs > force group = nobody > > I want only hdfs exported ;) > > read-performance with rsync: > rsync -av /test/hdfs/user/flume/weblogs/2011-12-07/ . > > sent 20478888644 bytes received 92606 bytes 17377158.46 bytes/sec > total size is 20475835998 > > (19GB, 16 MB/s) > > find . -type f |wc -l > 4665 > > plain over rsync -essh .... I get around 20mb/s, so I can live with. But > now the part I spent days without a solution: > > I perform a lot of write tests. I have a 10GB plain file (dd if of), thats > I use for all write tests. Now, if I try to copy the file from host1 (cifs > client), which hangs without socket options at 128MB. The interest part > there ist - 129MB are the blocksize in my hadoop-cluster. I played with > socket options and now the copy stops at 728MB. I get a nmb-error in > messages: > Dec 8 14:38:42 HOST2 nmbd[12609]: [2011/12/08 14:38:42, 0] > nmbd/nmbd.c:terminate(58) > Dec 8 14:38:42 HOST2 nmbd[12609]: Got SIGTERM: going down... > Dec 8 14:42:46 HOST2 kernel: INFO: task ls:12663 blocked for more than > 120 seconds. > > after the system freezes, HOST1 going into load 41 up. vmstat und iostat > say nothing, cp is going into state Z (zombie). Has anybody a advice? Could > be that the process is overloaded and die? > > A problem with fuse I could imagine, because if I copy the same file over > scp from HOST1 => HOST2 into hdfs I get a speed of 41 MB/s without any > errors. > > thanks, > - Alex > > -- > Alexander Lorenz > http://mapredit.blogspot.com > > *P **Think of the environment: please don't print this email unless you > really need to.* > > > -- Alexander Lorenz http://mapredit.blogspot.com *P **Think of the environment: please don't print this email unless you really need to.* -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba