Re: [Ocfs2-users] No space left on the device
Hi Aravind, Aravind Divakaran wrote: > Hi Tao, >> Hi Aravind, >> >> Aravind Divakaran wrote: Hi Aravind, Aravind Divakaran wrote: > Hi Tao, > >> Hi Aravind, >> >> Aravind Divakaran wrote: >>> Hi All, >>> >>> I have already sent one mail regarding the space issue i am facing >>> with >>> my >>> ocfs filesystem. As mentioned in the below link it is an issue >>> related >>> to >>> free space fragmentation. >>> >>> http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189 >>> >>> I have seen a patch for stealing extent allocation which was there >>> is >>> 2.6.34-rc1 kernel. So i compiled my new kernel and installed on my >>> system. >>> >>> Below is my ocfs details on my system >>> >>> #modinfo ocfs2 >>> >>> filename: /lib/modules/2.6.34-rc1/kernel/fs/ocfs2/ocfs2.ko >>> license:GPL >>> author: Oracle >>> version:1.5.0 >>> description:OCFS2 1.5.0 >>> srcversion: A8B69947E8FF56D74858993 >>> depends:jbd2,ocfs2_stackglue,quota_tree,ocfs2_nodemanager >>> vermagic: 2.6.34-rc1 SMP mod_unload modversions >>> >>> This is my stat_sysdir.sh output >>> >>> http://pastebin.com/RZH9DkTk >>> >>> Can anyone help me how to resolve this, please as the problem occurs >>> on >>> production mail server with 3000 emailid. >> I just checked your stat_sysdir output. It isn't caused by extent >> block >> alloc actually. So the patch doesn't work for you. Yes, the problem >> you >> meet is fragmentation issue, but the root cause is that inode_alloc >> can't allocate any more inodes(a little different from 1189). >> >> I am now working on discontiguous block group. It will resolve your >> issue I think. Hope it can be get into mainline in 2.6.35. >> >> Regards, >> Tao >> > For my previous mail i got reply from you > > "Another way is that you can cp the file to another volume, remove it > and > then cp back. It should be contiguous enough." > > As mentioned in the 1189 > > "However, reducing the slot count by 1 (to 4) may not be enough as it > does > not > have much contiguous space. It may work. But reducing it by 2 will > definitely work. > > Umount the volume on all nodes and run: > # tunefs.ocfs2 -N 3 /dev/sda1 > > Run fsck.ocfs2 for sanity checking." > > Will anyone of the above solution will temporary solve my problem. Yes, it works. I just replied you in another e-mail. Regards, Tao >>> I am running tunefs.ocfs2 on my 500gb harddisk which contain 215gb of >>> data, in order to reduce the slots. I had used the below command. >>> >>> tunefs.ocfs2 -N 3 /dev/mapper/store >>> >>> Now almost 7hours is over still it didnt finished the execution. Below >>> is >>> the output i am getting. >>> >>> node01:~# tunefs.ocfs2 -N 3 /dev/mapper/store >>> tunefs.ocfs2 1.4.1 >>> >>> How much time it will take to reduce the slots. Whether it will be >>> finished within 10hours. Can anyone help me. >> It shouldn't cost so much time. I guess it get blocked in some case. So >> is this volume umounted in all the nodes? If yes, could you please >> strace it to see what's wrong? >> >> Regards, >> Tao >> > > We have ocfs running with two cluster node. Before doing tunefs.ocfs2 i > have stopped my ocfs2 service in both the systems which will unmount my > ocfs filesystem. I have crossverified with mounted.ocfs2, which gives all > unmounted. I tried to strace also but my strace is also not giving any > output for that process id. Oh, so is the program still running? If yes, 'top' should show its state. If it is in 'D', please run ps -e -o pid,stat,comm,wchan=WIDE-WCHAN-COLUMN to see what's wrong. If it is in 'R', I guess there is a dead loop or somehow. anyway, could you please o2image/compress your disk image and then send to us? We can test it locally. See man o2image. Regards, Tao ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No space left on the device
Hi Aravind, Aravind Divakaran wrote: > Hi Tao, >> Hi Aravind, >> >> Aravind Divakaran wrote: Hi Aravind, Aravind Divakaran wrote: > Hi Tao, > >> Hi Aravind, >> >> Aravind Divakaran wrote: >>> Hi All, >>> >>> I have already sent one mail regarding the space issue i am facing >>> with >>> my >>> ocfs filesystem. As mentioned in the below link it is an issue >>> related >>> to >>> free space fragmentation. >>> >>> http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189 >>> >>> I have seen a patch for stealing extent allocation which was there >>> is >>> 2.6.34-rc1 kernel. So i compiled my new kernel and installed on my >>> system. >>> >>> Below is my ocfs details on my system >>> >>> #modinfo ocfs2 >>> >>> filename: /lib/modules/2.6.34-rc1/kernel/fs/ocfs2/ocfs2.ko >>> license:GPL >>> author: Oracle >>> version:1.5.0 >>> description:OCFS2 1.5.0 >>> srcversion: A8B69947E8FF56D74858993 >>> depends:jbd2,ocfs2_stackglue,quota_tree,ocfs2_nodemanager >>> vermagic: 2.6.34-rc1 SMP mod_unload modversions >>> >>> This is my stat_sysdir.sh output >>> >>> http://pastebin.com/RZH9DkTk >>> >>> Can anyone help me how to resolve this, please as the problem occurs >>> on >>> production mail server with 3000 emailid. >> I just checked your stat_sysdir output. It isn't caused by extent >> block >> alloc actually. So the patch doesn't work for you. Yes, the problem >> you >> meet is fragmentation issue, but the root cause is that inode_alloc >> can't allocate any more inodes(a little different from 1189). >> >> I am now working on discontiguous block group. It will resolve your >> issue I think. Hope it can be get into mainline in 2.6.35. >> >> Regards, >> Tao >> > For my previous mail i got reply from you > > "Another way is that you can cp the file to another volume, remove it > and > then cp back. It should be contiguous enough." > > As mentioned in the 1189 > > "However, reducing the slot count by 1 (to 4) may not be enough as it > does > not > have much contiguous space. It may work. But reducing it by 2 will > definitely work. > > Umount the volume on all nodes and run: > # tunefs.ocfs2 -N 3 /dev/sda1 > > Run fsck.ocfs2 for sanity checking." > > Will anyone of the above solution will temporary solve my problem. Yes, it works. I just replied you in another e-mail. Regards, Tao >>> I am running tunefs.ocfs2 on my 500gb harddisk which contain 215gb of >>> data, in order to reduce the slots. I had used the below command. >>> >>> tunefs.ocfs2 -N 3 /dev/mapper/store >>> >>> Now almost 7hours is over still it didnt finished the execution. Below >>> is >>> the output i am getting. >>> >>> node01:~# tunefs.ocfs2 -N 3 /dev/mapper/store >>> tunefs.ocfs2 1.4.1 >>> >>> How much time it will take to reduce the slots. Whether it will be >>> finished within 10hours. Can anyone help me. >> It shouldn't cost so much time. I guess it get blocked in some case. So >> is this volume umounted in all the nodes? If yes, could you please >> strace it to see what's wrong? >> >> Regards, >> Tao >> > > We have ocfs running with two cluster node. Before doing tunefs.ocfs2 i > have stopped my ocfs2 service in both the systems which will unmount my > ocfs filesystem. I have crossverified with mounted.ocfs2, which gives all > unmounted. I tried to strace also but my strace is also not giving any > output for that process id. Oh, so is the program still running? If yes, 'top' should show its state. If it is in 'D', please run ps -e -o pid,stat,comm,wchan=WIDE-WCHAN-COLUMN to see what's wrong. If it is in 'R', I guess there is a dead loop or somehow. anyway, could you please o2image/compress your disk image and then send to us? We can test it locally. See man o2image. Regards, Tao ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No space left on the device
Hi Tao, > Hi Aravind, > > Aravind Divakaran wrote: >>> Hi Aravind, >>> >>> Aravind Divakaran wrote: Hi Tao, > Hi Aravind, > > Aravind Divakaran wrote: >> Hi All, >> >> I have already sent one mail regarding the space issue i am facing >> with >> my >> ocfs filesystem. As mentioned in the below link it is an issue >> related >> to >> free space fragmentation. >> >> http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189 >> >> I have seen a patch for stealing extent allocation which was there >> is >> 2.6.34-rc1 kernel. So i compiled my new kernel and installed on my >> system. >> >> Below is my ocfs details on my system >> >> #modinfo ocfs2 >> >> filename: /lib/modules/2.6.34-rc1/kernel/fs/ocfs2/ocfs2.ko >> license:GPL >> author: Oracle >> version:1.5.0 >> description:OCFS2 1.5.0 >> srcversion: A8B69947E8FF56D74858993 >> depends:jbd2,ocfs2_stackglue,quota_tree,ocfs2_nodemanager >> vermagic: 2.6.34-rc1 SMP mod_unload modversions >> >> This is my stat_sysdir.sh output >> >> http://pastebin.com/RZH9DkTk >> >> Can anyone help me how to resolve this, please as the problem occurs >> on >> production mail server with 3000 emailid. > I just checked your stat_sysdir output. It isn't caused by extent > block > alloc actually. So the patch doesn't work for you. Yes, the problem > you > meet is fragmentation issue, but the root cause is that inode_alloc > can't allocate any more inodes(a little different from 1189). > > I am now working on discontiguous block group. It will resolve your > issue I think. Hope it can be get into mainline in 2.6.35. > > Regards, > Tao > For my previous mail i got reply from you "Another way is that you can cp the file to another volume, remove it and then cp back. It should be contiguous enough." As mentioned in the 1189 "However, reducing the slot count by 1 (to 4) may not be enough as it does not have much contiguous space. It may work. But reducing it by 2 will definitely work. Umount the volume on all nodes and run: # tunefs.ocfs2 -N 3 /dev/sda1 Run fsck.ocfs2 for sanity checking." Will anyone of the above solution will temporary solve my problem. >>> Yes, it works. I just replied you in another e-mail. >>> >>> Regards, >>> Tao >>> >> I am running tunefs.ocfs2 on my 500gb harddisk which contain 215gb of >> data, in order to reduce the slots. I had used the below command. >> >> tunefs.ocfs2 -N 3 /dev/mapper/store >> >> Now almost 7hours is over still it didnt finished the execution. Below >> is >> the output i am getting. >> >> node01:~# tunefs.ocfs2 -N 3 /dev/mapper/store >> tunefs.ocfs2 1.4.1 >> >> How much time it will take to reduce the slots. Whether it will be >> finished within 10hours. Can anyone help me. > It shouldn't cost so much time. I guess it get blocked in some case. So > is this volume umounted in all the nodes? If yes, could you please > strace it to see what's wrong? > > Regards, > Tao > We have ocfs running with two cluster node. Before doing tunefs.ocfs2 i have stopped my ocfs2 service in both the systems which will unmount my ocfs filesystem. I have crossverified with mounted.ocfs2, which gives all unmounted. I tried to strace also but my strace is also not giving any output for that process id. Rgds, Aravind M D ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No space left on the device
Hi Aravind, Aravind Divakaran wrote: >> Hi Aravind, >> >> Aravind Divakaran wrote: >>> Hi Tao, >>> Hi Aravind, Aravind Divakaran wrote: > Hi All, > > I have already sent one mail regarding the space issue i am facing > with > my > ocfs filesystem. As mentioned in the below link it is an issue related > to > free space fragmentation. > > http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189 > > I have seen a patch for stealing extent allocation which was there is > 2.6.34-rc1 kernel. So i compiled my new kernel and installed on my > system. > > Below is my ocfs details on my system > > #modinfo ocfs2 > > filename: /lib/modules/2.6.34-rc1/kernel/fs/ocfs2/ocfs2.ko > license:GPL > author: Oracle > version:1.5.0 > description:OCFS2 1.5.0 > srcversion: A8B69947E8FF56D74858993 > depends:jbd2,ocfs2_stackglue,quota_tree,ocfs2_nodemanager > vermagic: 2.6.34-rc1 SMP mod_unload modversions > > This is my stat_sysdir.sh output > > http://pastebin.com/RZH9DkTk > > Can anyone help me how to resolve this, please as the problem occurs > on > production mail server with 3000 emailid. I just checked your stat_sysdir output. It isn't caused by extent block alloc actually. So the patch doesn't work for you. Yes, the problem you meet is fragmentation issue, but the root cause is that inode_alloc can't allocate any more inodes(a little different from 1189). I am now working on discontiguous block group. It will resolve your issue I think. Hope it can be get into mainline in 2.6.35. Regards, Tao >>> For my previous mail i got reply from you >>> >>> "Another way is that you can cp the file to another volume, remove it >>> and >>> then cp back. It should be contiguous enough." >>> >>> As mentioned in the 1189 >>> >>> "However, reducing the slot count by 1 (to 4) may not be enough as it >>> does >>> not >>> have much contiguous space. It may work. But reducing it by 2 will >>> definitely work. >>> >>> Umount the volume on all nodes and run: >>> # tunefs.ocfs2 -N 3 /dev/sda1 >>> >>> Run fsck.ocfs2 for sanity checking." >>> >>> Will anyone of the above solution will temporary solve my problem. >> Yes, it works. I just replied you in another e-mail. >> >> Regards, >> Tao >> > I am running tunefs.ocfs2 on my 500gb harddisk which contain 215gb of > data, in order to reduce the slots. I had used the below command. > > tunefs.ocfs2 -N 3 /dev/mapper/store > > Now almost 7hours is over still it didnt finished the execution. Below is > the output i am getting. > > node01:~# tunefs.ocfs2 -N 3 /dev/mapper/store > tunefs.ocfs2 1.4.1 > > How much time it will take to reduce the slots. Whether it will be > finished within 10hours. Can anyone help me. It shouldn't cost so much time. I guess it get blocked in some case. So is this volume umounted in all the nodes? If yes, could you please strace it to see what's wrong? Regards, Tao ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No space left on the device
Hi Aravind, Aravind Divakaran wrote: > Hi Tao, >>> Hi Aravind, >>> >>> Aravind Divakaran wrote: Hi Tao, > Hi Aravind, > > Aravind Divakaran wrote: >> Hi All, > > After running the tunefs.ocfs2 command i am getting the following error on > my console > > node01#tunefs.ocfs2 -N 2 /dev/mapper/store > tunefs.ocfs2 1.4.1 > Segmentation fault > node01:~# > Message from sysl...@node01 at Mar 19 05:25:15 ... > kernel:[ 709.834536] [ cut here ] > > Message from sysl...@node01 at Mar 19 05:25:15 ... > kernel:[ 709.834678] invalid opcode: [#1] SMP > > Message from sysl...@node01 at Mar 19 05:25:15 ... > kernel:[ 709.834820] last sysfs file: /sys/fs/o2cb/interface_revision > > Message from sysl...@node01 at Mar 19 05:25:15 ... > kernel:[ 709.838490] Stack: > > Message from sysl...@node01 at Mar 19 05:25:15 ... > kernel:[ 709.838490] Call Trace: > > Message from sysl...@node01 at Mar 19 05:25:15 ... > kernel:[ 709.838490] Code: 00 00 80 00 f7 c7 00 00 04 00 74 0b 81 e7 ff > ff fb ff 0d 00 00 02 00 f7 c7 0 > 0f> 0b eb fe c3 48 8b 47 58 48 8b 40 48 4c 8b 58 08 41 ff e3 48" > > And my /var/log/messages shows this error > > Mar 19 05:25:15 cmnode01 kernel: [ 709.837837] > Mar 19 05:25:15 cmnode01 kernel: [ 709.837896] Pid: 9051, comm: > tunefs.ocfs2 Not tainted 2.6.34-rc1 #1 S > Mar 19 05:25:15 cmnode01 kernel: [ 709.837984] RIP: > 0010:[] [] flag > Mar 19 05:25:15 cmnode01 kernel: [ 709.838115] RSP: 0018:8802aadc7bc0 > EFLAGS: 00010206 > Mar 19 05:25:15 cmnode01 kernel: [ 709.838179] RAX: 0100 RBX: > 001f RCX: > Mar 19 05:25:15 cmnode01 kernel: [ 709.838246] RDX: 8802a90d6700 RSI: > 0005 RDI: > Mar 19 05:25:15 cmnode01 kernel: [ 709.838313] RBP: 8802a90d6700 R08: > 8802a90d66d0 R09: > Mar 19 05:25:15 cmnode01 kernel: [ 709.838381] R10: dead00100100 R11: > a0297143 R12: 8802 > Mar 19 05:25:15 cmnode01 kernel: [ 709.838448] R13: 0005 R14: > 8802a90d66d0 R15: 8802 > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] FS: > 7fd97b54b760() GS:88000184() knl > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] CS: 0010 DS: ES: > CR0: 80050033 > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] CR2: 01c54048 CR3: > 00028798c000 CR4: > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] DR0: DR1: > DR2: > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] DR3: DR6: > 0ff0 DR7: > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] Process tunefs.ocfs2 (pid: > 9051, threadinfo 8802aadc6 > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] a029716e > 0001 0286 8 > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] <0> 8802ae72f9e8 > 8802a90d66c8 0005 00 > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] <0> 8802aadc7c78 > 8802aadc7c90 a029e274 00 > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] [] ? > o2cb_dlm_lock+0x2b/0x78 [ocfs2_st > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] [] ? > user_dlm_cluster_lock+0x2f7/0x44d > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] [] ? > __blockdev_direct_IO+0x93e/0x996 > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] [] ? > dlmfs_file_open+0x0/0x17d [ocfs2_ > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] [] ? > dlmfs_file_open+0x141/0x17d [ocfs > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] [] ? > inotify_d_instantiate+0x12/0x38 > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] [] ? > dlmfs_file_open+0x0/0x17d [ocfs2_ > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] [] ? > __dentry_open+0x17f/0x2a1 > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] [] ? > do_last+0x3a8/0x644 > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] [] ? > do_filp_open+0x1ed/0x5f2 > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] [] ? > handle_mm_fault+0x3ee/0x876 > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] [] ? > touch_atime+0x7c/0x127 > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] [] ? > do_sys_open+0x55/0xfc > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] [] ? > system_call_fastpath+0x16/0x1b > Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] RSP > > Can you please help me why i am getting this error. It looks that you meet with a kernel panic. Strange. So do you umount the volume before running tunefs.ocfs2? We have never meet with it before. Could you please file a bug in http://oss.oracle.com/bugzilla/ about this. Thanks. The good thing is that it corrupts in dlmfs, so your data should be ok. Regards, Tao ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No space left on the device
Hi Tao, > >> Hi Aravind, >> >> Aravind Divakaran wrote: >>> Hi Tao, >>> Hi Aravind, Aravind Divakaran wrote: > Hi All, > > I have already sent one mail regarding the space issue i am facing > with > my > ocfs filesystem. As mentioned in the below link it is an issue > related > to > free space fragmentation. > > http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189 > > I have seen a patch for stealing extent allocation which was there is > 2.6.34-rc1 kernel. So i compiled my new kernel and installed on my > system. > > Below is my ocfs details on my system > > #modinfo ocfs2 > > filename: /lib/modules/2.6.34-rc1/kernel/fs/ocfs2/ocfs2.ko > license:GPL > author: Oracle > version:1.5.0 > description:OCFS2 1.5.0 > srcversion: A8B69947E8FF56D74858993 > depends:jbd2,ocfs2_stackglue,quota_tree,ocfs2_nodemanager > vermagic: 2.6.34-rc1 SMP mod_unload modversions > > This is my stat_sysdir.sh output > > http://pastebin.com/RZH9DkTk > > Can anyone help me how to resolve this, please as the problem occurs > on > production mail server with 3000 emailid. I just checked your stat_sysdir output. It isn't caused by extent block alloc actually. So the patch doesn't work for you. Yes, the problem you meet is fragmentation issue, but the root cause is that inode_alloc can't allocate any more inodes(a little different from 1189). I am now working on discontiguous block group. It will resolve your issue I think. Hope it can be get into mainline in 2.6.35. Regards, Tao >>> >>> For my previous mail i got reply from you >>> >>> "Another way is that you can cp the file to another volume, remove it >>> and >>> then cp back. It should be contiguous enough." >>> >>> As mentioned in the 1189 >>> >>> "However, reducing the slot count by 1 (to 4) may not be enough as it >>> does >>> not >>> have much contiguous space. It may work. But reducing it by 2 will >>> definitely work. >>> >>> Umount the volume on all nodes and run: >>> # tunefs.ocfs2 -N 3 /dev/sda1 >>> >>> Run fsck.ocfs2 for sanity checking." >>> >>> Will anyone of the above solution will temporary solve my problem. >> Yes, it works. I just replied you in another e-mail. >> >> Regards, >> Tao >> > I am running tunefs.ocfs2 on my 500gb harddisk which contain 215gb of > data, in order to reduce the slots. I had used the below command. > > tunefs.ocfs2 -N 3 /dev/mapper/store > > Now almost 7hours is over still it didnt finished the execution. Below is > the output i am getting. > > node01:~# tunefs.ocfs2 -N 3 /dev/mapper/store > tunefs.ocfs2 1.4.1 > > How much time it will take to reduce the slots. Whether it will be > finished within 10hours. Can anyone help me. > > Rgds, > > Aravind M D > > > > ___ > Ocfs2-users mailing list > Ocfs2-users@oss.oracle.com > http://oss.oracle.com/mailman/listinfo/ocfs2-users > After running the tunefs.ocfs2 command i am getting the following error on my console node01#tunefs.ocfs2 -N 2 /dev/mapper/store tunefs.ocfs2 1.4.1 Segmentation fault node01:~# Message from sysl...@node01 at Mar 19 05:25:15 ... kernel:[ 709.834536] [ cut here ] Message from sysl...@node01 at Mar 19 05:25:15 ... kernel:[ 709.834678] invalid opcode: [#1] SMP Message from sysl...@node01 at Mar 19 05:25:15 ... kernel:[ 709.834820] last sysfs file: /sys/fs/o2cb/interface_revision Message from sysl...@node01 at Mar 19 05:25:15 ... kernel:[ 709.838490] Stack: Message from sysl...@node01 at Mar 19 05:25:15 ... kernel:[ 709.838490] Call Trace: Message from sysl...@node01 at Mar 19 05:25:15 ... kernel:[ 709.838490] Code: 00 00 80 00 f7 c7 00 00 04 00 74 0b 81 e7 ff ff fb ff 0d 00 00 02 00 f7 c7 0 0f> 0b eb fe c3 48 8b 47 58 48 8b 40 48 4c 8b 58 08 41 ff e3 48" And my /var/log/messages shows this error Mar 19 05:25:15 cmnode01 kernel: [ 709.837837] Mar 19 05:25:15 cmnode01 kernel: [ 709.837896] Pid: 9051, comm: tunefs.ocfs2 Not tainted 2.6.34-rc1 #1 S Mar 19 05:25:15 cmnode01 kernel: [ 709.837984] RIP: 0010:[] [] flag Mar 19 05:25:15 cmnode01 kernel: [ 709.838115] RSP: 0018:8802aadc7bc0 EFLAGS: 00010206 Mar 19 05:25:15 cmnode01 kernel: [ 709.838179] RAX: 0100 RBX: 001f RCX: Mar 19 05:25:15 cmnode01 kernel: [ 709.838246] RDX: 8802a90d6700 RSI: 0005 RDI: Mar 19 05:25:15 cmnode01 kernel: [ 709.838313] RBP: 8802a90d6700 R08: 8802a90d66d0 R09: Mar 19 05:25:15 cmnode01 kernel: [ 709.838381] R10: dead00100100 R11: a0297143 R12: 8802 Mar 19 05:25:15 cmnode01 kernel: [ 709.838448] R13: 0005 R14: 8802a90d66d0 R15: 8802 Mar 19 05:25:15 cmnode01 kernel: [ 709.838490] FS: 7fd97b54b760() GS:8800018400
Re: [Ocfs2-users] No space left on the device
> Hi Aravind, > > Aravind Divakaran wrote: >> Hi Tao, >> >>> Hi Aravind, >>> >>> Aravind Divakaran wrote: Hi All, I have already sent one mail regarding the space issue i am facing with my ocfs filesystem. As mentioned in the below link it is an issue related to free space fragmentation. http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189 I have seen a patch for stealing extent allocation which was there is 2.6.34-rc1 kernel. So i compiled my new kernel and installed on my system. Below is my ocfs details on my system #modinfo ocfs2 filename: /lib/modules/2.6.34-rc1/kernel/fs/ocfs2/ocfs2.ko license:GPL author: Oracle version:1.5.0 description:OCFS2 1.5.0 srcversion: A8B69947E8FF56D74858993 depends:jbd2,ocfs2_stackglue,quota_tree,ocfs2_nodemanager vermagic: 2.6.34-rc1 SMP mod_unload modversions This is my stat_sysdir.sh output http://pastebin.com/RZH9DkTk Can anyone help me how to resolve this, please as the problem occurs on production mail server with 3000 emailid. >>> I just checked your stat_sysdir output. It isn't caused by extent block >>> alloc actually. So the patch doesn't work for you. Yes, the problem you >>> meet is fragmentation issue, but the root cause is that inode_alloc >>> can't allocate any more inodes(a little different from 1189). >>> >>> I am now working on discontiguous block group. It will resolve your >>> issue I think. Hope it can be get into mainline in 2.6.35. >>> >>> Regards, >>> Tao >>> >> >> For my previous mail i got reply from you >> >> "Another way is that you can cp the file to another volume, remove it >> and >> then cp back. It should be contiguous enough." >> >> As mentioned in the 1189 >> >> "However, reducing the slot count by 1 (to 4) may not be enough as it >> does >> not >> have much contiguous space. It may work. But reducing it by 2 will >> definitely work. >> >> Umount the volume on all nodes and run: >> # tunefs.ocfs2 -N 3 /dev/sda1 >> >> Run fsck.ocfs2 for sanity checking." >> >> Will anyone of the above solution will temporary solve my problem. > Yes, it works. I just replied you in another e-mail. > > Regards, > Tao > I am running tunefs.ocfs2 on my 500gb harddisk which contain 215gb of data, in order to reduce the slots. I had used the below command. tunefs.ocfs2 -N 3 /dev/mapper/store Now almost 7hours is over still it didnt finished the execution. Below is the output i am getting. node01:~# tunefs.ocfs2 -N 3 /dev/mapper/store tunefs.ocfs2 1.4.1 How much time it will take to reduce the slots. Whether it will be finished within 10hours. Can anyone help me. Rgds, Aravind M D ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No space left on the device
Hi Aravind, Aravind Divakaran wrote: > Hi Tao, > >> Hi Aravind, >> >> Aravind Divakaran wrote: >>> Hi All, >>> >>> I have already sent one mail regarding the space issue i am facing with >>> my >>> ocfs filesystem. As mentioned in the below link it is an issue related >>> to >>> free space fragmentation. >>> >>> http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189 >>> >>> I have seen a patch for stealing extent allocation which was there is >>> 2.6.34-rc1 kernel. So i compiled my new kernel and installed on my >>> system. >>> >>> Below is my ocfs details on my system >>> >>> #modinfo ocfs2 >>> >>> filename: /lib/modules/2.6.34-rc1/kernel/fs/ocfs2/ocfs2.ko >>> license:GPL >>> author: Oracle >>> version:1.5.0 >>> description:OCFS2 1.5.0 >>> srcversion: A8B69947E8FF56D74858993 >>> depends:jbd2,ocfs2_stackglue,quota_tree,ocfs2_nodemanager >>> vermagic: 2.6.34-rc1 SMP mod_unload modversions >>> >>> This is my stat_sysdir.sh output >>> >>> http://pastebin.com/RZH9DkTk >>> >>> Can anyone help me how to resolve this, please as the problem occurs on >>> production mail server with 3000 emailid. >> I just checked your stat_sysdir output. It isn't caused by extent block >> alloc actually. So the patch doesn't work for you. Yes, the problem you >> meet is fragmentation issue, but the root cause is that inode_alloc >> can't allocate any more inodes(a little different from 1189). >> >> I am now working on discontiguous block group. It will resolve your >> issue I think. Hope it can be get into mainline in 2.6.35. >> >> Regards, >> Tao >> > > For my previous mail i got reply from you > > "Another way is that you can cp the file to another volume, remove it and > then cp back. It should be contiguous enough." > > As mentioned in the 1189 > > "However, reducing the slot count by 1 (to 4) may not be enough as it does > not > have much contiguous space. It may work. But reducing it by 2 will > definitely work. > > Umount the volume on all nodes and run: > # tunefs.ocfs2 -N 3 /dev/sda1 > > Run fsck.ocfs2 for sanity checking." > > Will anyone of the above solution will temporary solve my problem. Yes, it works. I just replied you in another e-mail. Regards, Tao ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No space left on the device
Hi Tao, > Hi Aravind, > > Aravind Divakaran wrote: >> Hi All, >> >> I have already sent one mail regarding the space issue i am facing with >> my >> ocfs filesystem. As mentioned in the below link it is an issue related >> to >> free space fragmentation. >> >> http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189 >> >> I have seen a patch for stealing extent allocation which was there is >> 2.6.34-rc1 kernel. So i compiled my new kernel and installed on my >> system. >> >> Below is my ocfs details on my system >> >> #modinfo ocfs2 >> >> filename: /lib/modules/2.6.34-rc1/kernel/fs/ocfs2/ocfs2.ko >> license:GPL >> author: Oracle >> version:1.5.0 >> description:OCFS2 1.5.0 >> srcversion: A8B69947E8FF56D74858993 >> depends:jbd2,ocfs2_stackglue,quota_tree,ocfs2_nodemanager >> vermagic: 2.6.34-rc1 SMP mod_unload modversions >> >> This is my stat_sysdir.sh output >> >> http://pastebin.com/RZH9DkTk >> >> Can anyone help me how to resolve this, please as the problem occurs on >> production mail server with 3000 emailid. > I just checked your stat_sysdir output. It isn't caused by extent block > alloc actually. So the patch doesn't work for you. Yes, the problem you > meet is fragmentation issue, but the root cause is that inode_alloc > can't allocate any more inodes(a little different from 1189). > > I am now working on discontiguous block group. It will resolve your > issue I think. Hope it can be get into mainline in 2.6.35. > > Regards, > Tao > For my previous mail i got reply from you "Another way is that you can cp the file to another volume, remove it and then cp back. It should be contiguous enough." As mentioned in the 1189 "However, reducing the slot count by 1 (to 4) may not be enough as it does not have much contiguous space. It may work. But reducing it by 2 will definitely work. Umount the volume on all nodes and run: # tunefs.ocfs2 -N 3 /dev/sda1 Run fsck.ocfs2 for sanity checking." Will anyone of the above solution will temporary solve my problem. Rgds, Aravind M D ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No space left on the device
Hi Aravind, Aravind Divakaran wrote: > Hi All, > > I have already sent one mail regarding the space issue i am facing with my > ocfs filesystem. As mentioned in the below link it is an issue related to > free space fragmentation. > > http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189 > > I have seen a patch for stealing extent allocation which was there is > 2.6.34-rc1 kernel. So i compiled my new kernel and installed on my system. > > Below is my ocfs details on my system > > #modinfo ocfs2 > > filename: /lib/modules/2.6.34-rc1/kernel/fs/ocfs2/ocfs2.ko > license:GPL > author: Oracle > version:1.5.0 > description:OCFS2 1.5.0 > srcversion: A8B69947E8FF56D74858993 > depends:jbd2,ocfs2_stackglue,quota_tree,ocfs2_nodemanager > vermagic: 2.6.34-rc1 SMP mod_unload modversions > > This is my stat_sysdir.sh output > > http://pastebin.com/RZH9DkTk > > Can anyone help me how to resolve this, please as the problem occurs on > production mail server with 3000 emailid. I just checked your stat_sysdir output. It isn't caused by extent block alloc actually. So the patch doesn't work for you. Yes, the problem you meet is fragmentation issue, but the root cause is that inode_alloc can't allocate any more inodes(a little different from 1189). I am now working on discontiguous block group. It will resolve your issue I think. Hope it can be get into mainline in 2.6.35. Regards, Tao ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
[Ocfs2-users] No space left on the device
Hi All, I have already sent one mail regarding the space issue i am facing with my ocfs filesystem. As mentioned in the below link it is an issue related to free space fragmentation. http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189 I have seen a patch for stealing extent allocation which was there is 2.6.34-rc1 kernel. So i compiled my new kernel and installed on my system. Below is my ocfs details on my system #modinfo ocfs2 filename: /lib/modules/2.6.34-rc1/kernel/fs/ocfs2/ocfs2.ko license:GPL author: Oracle version:1.5.0 description:OCFS2 1.5.0 srcversion: A8B69947E8FF56D74858993 depends:jbd2,ocfs2_stackglue,quota_tree,ocfs2_nodemanager vermagic: 2.6.34-rc1 SMP mod_unload modversions This is my stat_sysdir.sh output http://pastebin.com/RZH9DkTk Can anyone help me how to resolve this, please as the problem occurs on production mail server with 3000 emailid. Rgds, Aravind M D ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No Space left on the device.
Hi Joel, My ocfs version is 1.4.1 and my kernel version is 2.6.30. Rgds, Aravind M D > On Fri, Mar 05, 2010 at 10:23:19AM +0530, Aravind Divakaran wrote: >> My ocfs filesystem has 270gb free space. > > What version of ocfs2 do you have? What kernel is it running > on? > > Joel > > -- > > "I'm so tired of being tired, > Sure as night will follow day. > Most things I worry about > Never happen anyway." > > Joel Becker > Principal Software Developer > Oracle > E-mail: joel.bec...@oracle.com > Phone: (650) 506-8127 > ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No Space left on the device.
On Fri, Mar 05, 2010 at 04:12:51PM -0600, Brian Kroth wrote: > Fair enough. Here's the most "authoritative" source [1] [2] I can find, > though a quick google on "long term stable kernel" produces a number of > other results [3]. > > [1] http://lwn.net/Articles/370236/ > [2] http://www.kroah.com/log/linux/stable-status-01-2010.html > [3] http://www.fabian-fingerle.de/2010-02-23.233 Ok, its sta...@kernel.org. We'll see what comes up. Joel -- "If at first you don't succeed, cover all traces that you tried." -Unknown Joel Becker Principal Software Developer Oracle E-mail: joel.bec...@oracle.com Phone: (650) 506-8127 ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No Space left on the device.
Joel Becker 2010-03-05 13:48: > On Fri, Mar 05, 2010 at 08:33:34AM -0600, Brian Kroth wrote: > > As mentioned in the bug (didn't think it was a proper place for > > discussion) I'm also curious more generally about backporting these > > fixes to the 2.6.32 kernel since it's been designated long term stable. > > Is that responsibility just on the individual distro's kernel maintainer > > or are the OCFS2 devs planning on submitting fixes to the mainline > > 2.6.32 tree? > > Who 'designated' it long term stable? I'm just wondering who we > should send our patches to ;-) > > Joel Fair enough. Here's the most "authoritative" source [1] [2] I can find, though a quick google on "long term stable kernel" produces a number of other results [3]. [1] http://lwn.net/Articles/370236/ [2] http://www.kroah.com/log/linux/stable-status-01-2010.html [3] http://www.fabian-fingerle.de/2010-02-23.233 Thanks, Brian signature.asc Description: Digital signature ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No Space left on the device.
On Fri, Mar 05, 2010 at 10:23:19AM +0530, Aravind Divakaran wrote: > My ocfs filesystem has 270gb free space. What version of ocfs2 do you have? What kernel is it running on? Joel -- "I'm so tired of being tired, Sure as night will follow day. Most things I worry about Never happen anyway." Joel Becker Principal Software Developer Oracle E-mail: joel.bec...@oracle.com Phone: (650) 506-8127 ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No Space left on the device.
On Fri, Mar 05, 2010 at 08:33:34AM -0600, Brian Kroth wrote: > As mentioned in the bug (didn't think it was a proper place for > discussion) I'm also curious more generally about backporting these > fixes to the 2.6.32 kernel since it's been designated long term stable. > Is that responsibility just on the individual distro's kernel maintainer > or are the OCFS2 devs planning on submitting fixes to the mainline > 2.6.32 tree? Who 'designated' it long term stable? I'm just wondering who we should send our patches to ;-) Joel -- Life's Little Instruction Book #232 "Keep your promises." Joel Becker Principal Software Developer Oracle E-mail: joel.bec...@oracle.com Phone: (650) 506-8127 ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No Space left on the device.
I also have a mail volume hosted on OCFS2 and I'm somewhat concerned about /when/ we will run into this problem and what we can do to help avoid too much hurt when it happens. Are there any tips on reading the output of stat_sysdir.sh? The man page wasn't especially helpful, but I'm guessing I'm looking at the Contig column for "enough" clusters > 511. I can post the output if you'd prefer. As mentioned in the bug (didn't think it was a proper place for discussion) I'm also curious more generally about backporting these fixes to the 2.6.32 kernel since it's been designated long term stable. Is that responsibility just on the individual distro's kernel maintainer or are the OCFS2 devs planning on submitting fixes to the mainline 2.6.32 tree? Thanks, Brian Brad Plant 2010-03-04 16:17: > Hi Aravind, > > Sounds like you might have hit the free space fragmentation issue: > http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189 > > I'm sure that if you post output of stat_sysdir.sh > (http://oss.oracle.com/~seeda/misc/stat_sysdir.sh) one of the ocfs2 > devs will be able to confirm this. > > *If* it is this problem, removing some node slots will help. That is > of course if you have more node slots that you need. I think 8 are > created by default. > > Cheers, > > Brad > > > On Thu, 4 Mar 2010 10:28:49 +0530 (IST) > "Aravind Divakaran" wrote: > > > HiAll, > > > > For my mailserver i am using ocfs2 filesystem configured on san. Now my > > mail delivery application is sometimes complaining No Space left on the > > device, even though there is enough space and inodes. Can anyone help me > > to solve this issue. > > > > > > Rgds, > > Aravind M D > > > > > > > > ___ > > Ocfs2-users mailing list > > Ocfs2-users@oss.oracle.com > > http://oss.oracle.com/mailman/listinfo/ocfs2-users signature.asc Description: Digital signature ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No Space left on the device.
Brad Plant wrote: > Hi Tao, > > On Fri, 05 Mar 2010 15:03:50 +0800 > Tao Ma wrote: > >>> Assuming we *can* still write to the FS (i.e. as more of a preventative >>> action), would the following do the same? >>> >>> cp -a a b >>> mv b a >>> >>> Can the above work as a *hack* online defrag? Will this reduce the amount >>> of free space fragmentation and therefore stop (or reduce the chance of) >>> this problem from occurring? Would we need to do it to the entire file >>> system or just a few larger files? >> yeah, that should work. >> you can use debugfs.ocfs2 to see whether a file is more fragmented or not. > > Ok, so it will defrag files, but will it help with the inability to write to > the file system issue reported in > http://oss.oracle.com/bugzilla/show_bugcgi?id=1189 ? yes, it helps to defrag files by cp. But the root cause for bug 1189 is that the old mechanism is somehow making the volume more fragmented. So the real solution has to be a bug fix in kernel, which like Sunil said, is ongoing. Regards, Tao ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No Space left on the device.
Hi Tao, On Fri, 05 Mar 2010 15:03:50 +0800 Tao Ma wrote: > > Assuming we *can* still write to the FS (i.e. as more of a preventative > > action), would the following do the same? > > > > cp -a a b > > mv b a > > > > Can the above work as a *hack* online defrag? Will this reduce the amount > > of free space fragmentation and therefore stop (or reduce the chance of) > > this problem from occurring? Would we need to do it to the entire file > > system or just a few larger files? > yeah, that should work. > you can use debugfs.ocfs2 to see whether a file is more fragmented or not. Ok, so it will defrag files, but will it help with the inability to write to the file system issue reported in http://oss.oracle.com/bugzilla/show_bugcgi?id=1189 ? Cheers, Brad signature.asc Description: PGP signature ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No Space left on the device.
Hi Brad, Brad Plant wrote: > Hi Tao, > > On Fri, 05 Mar 2010 14:33:36 +0800 > Tao Ma wrote: > >> Another way is that you can cp the file to another volume, remove it and >> then cp back. It should be contiguous enough. > > Assuming we *can* still write to the FS (i.e. as more of a preventative > action), would the following do the same? > > cp -a a b > mv b a > > Can the above work as a *hack* online defrag? Will this reduce the amount of > free space fragmentation and therefore stop (or reduce the chance of) this > problem from occurring? Would we need to do it to the entire file system or > just a few larger files? yeah, that should work. you can use debugfs.ocfs2 to see whether a file is more fragmented or not. Regards, Tao > > Cheers, > > Brad ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No Space left on the device.
Hi Tao, On Fri, 05 Mar 2010 14:33:36 +0800 Tao Ma wrote: > Another way is that you can cp the file to another volume, remove it and > then cp back. It should be contiguous enough. Assuming we *can* still write to the FS (i.e. as more of a preventative action), would the following do the same? cp -a a b mv b a Can the above work as a *hack* online defrag? Will this reduce the amount of free space fragmentation and therefore stop (or reduce the chance of) this problem from occurring? Would we need to do it to the entire file system or just a few larger files? Cheers, Brad signature.asc Description: PGP signature ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No Space left on the device.
Hi Aravind, Aravind Divakaran wrote: > Hi, > > I am facing problem due to free space fragmentation. > > http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189. > > In the above link it is mentioned that reducing the slots can solve the > issue. Right now i have 4slots. As my ocfs device is configured for a > failover for mailserver of 2systems, if i reduce it to 2slots whether i > can solve this issue. Please let me know if this is the command i have to > give to reduce the slots to 2. Decreasing slots just release the volume we reserved for journal, so it didn't help for free space fragmentation actually. I have looked into your system_dir output, the journal size is only 256M, so in total you will have 512M more free space. Don't know whether it is big enough for you. > > tunefs.ocfs2 -N 2 /dev/mapper/store This is an offline action, so you need to umount the volume from all the nodes to go ahead. Another way is that you can cp the file to another volume, remove it and then cp back. It should be contiguous enough. Regards, Tao > > Rgds, > Aravind M D > >> Hi Aravind, >> >> Aravind Divakaran wrote: >>> Hi >>> >>> My ocfs filesystem has 270gb free space. >>> >>> FilesystemSize Used Avail Use% Mounted on >>> /dev/mapper/store 501G 232G 270G 47% /data >>> >>> INode details for ocfs filesystem is >>> >>> FilesystemInodes IUsed IFree IUse% Mounted on >>> /dev/mapper/store131225616 60588258 70637358 47% /data >>> >>> I have run the stat_sysdir.sh on /dev/mapper/store. The output is on the >>> below link. >>> >>> http://pastebin.com/4VLLCpRa >> I just went through the output and confirmed that the culprit is free >> space fragmentation. So please refer to >> http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189. >> >> Regards, >> Tao >>> Please help me to resolve this issue. >>> >>> Rgds, >>> Aravind M D >>> >>> >>> Hi Aravind, Sounds like you might have hit the free space fragmentation issue: http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189 I'm sure that if you post output of stat_sysdir.sh (http://oss.oracle.com/~seeda/misc/stat_sysdir.sh) one of the ocfs2 devs will be able to confirm this. *If* it is this problem, removing some node slots will help. That is of course if you have more node slots that you need. I think 8 are created by default. Cheers, Brad On Thu, 4 Mar 2010 10:28:49 +0530 (IST) "Aravind Divakaran" wrote: > HiAll, > > For my mailserver i am using ocfs2 filesystem configured on san. Now > my > mail delivery application is sometimes complaining No Space left on > the > device, even though there is enough space and inodes. Can anyone help > me > to solve this issue. > > > Rgds, > Aravind M D > > > > ___ > Ocfs2-users mailing list > Ocfs2-users@oss.oracle.com > http://oss.oracle.com/mailman/listinfo/ocfs2-users ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users >>> >>> >>> >>> >>> ___ >>> Ocfs2-users mailing list >>> Ocfs2-users@oss.oracle.com >>> http://oss.oracle.com/mailman/listinfo/ocfs2-users > > ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No Space left on the device.
Hi, I am facing problem due to free space fragmentation. http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189. In the above link it is mentioned that reducing the slots can solve the issue. Right now i have 4slots. As my ocfs device is configured for a failover for mailserver of 2systems, if i reduce it to 2slots whether i can solve this issue. Please let me know if this is the command i have to give to reduce the slots to 2. tunefs.ocfs2 -N 2 /dev/mapper/store Rgds, Aravind M D > Hi Aravind, > > Aravind Divakaran wrote: >> Hi >> >> My ocfs filesystem has 270gb free space. >> >> FilesystemSize Used Avail Use% Mounted on >> /dev/mapper/store 501G 232G 270G 47% /data >> >> INode details for ocfs filesystem is >> >> FilesystemInodes IUsed IFree IUse% Mounted on >> /dev/mapper/store131225616 60588258 70637358 47% /data >> >> I have run the stat_sysdir.sh on /dev/mapper/store. The output is on the >> below link. >> >> http://pastebin.com/4VLLCpRa > I just went through the output and confirmed that the culprit is free > space fragmentation. So please refer to > http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189. > > Regards, > Tao >> >> Please help me to resolve this issue. >> >> Rgds, >> Aravind M D >> >> >> >>> Hi Aravind, >>> >>> Sounds like you might have hit the free space fragmentation issue: >>> http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189 >>> >>> I'm sure that if you post output of stat_sysdir.sh >>> (http://oss.oracle.com/~seeda/misc/stat_sysdir.sh) one of the ocfs2 >>> devs >>> will be able to confirm this. >>> >>> *If* it is this problem, removing some node slots will help. That is of >>> course if you have more node slots that you need. I think 8 are created >>> by >>> default. >>> >>> Cheers, >>> >>> Brad >>> >>> >>> On Thu, 4 Mar 2010 10:28:49 +0530 (IST) >>> "Aravind Divakaran" wrote: >>> HiAll, For my mailserver i am using ocfs2 filesystem configured on san. Now my mail delivery application is sometimes complaining No Space left on the device, even though there is enough space and inodes. Can anyone help me to solve this issue. Rgds, Aravind M D ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users >>> ___ >>> Ocfs2-users mailing list >>> Ocfs2-users@oss.oracle.com >>> http://oss.oracle.com/mailman/listinfo/ocfs2-users >> >> >> >> >> >> ___ >> Ocfs2-users mailing list >> Ocfs2-users@oss.oracle.com >> http://oss.oracle.com/mailman/listinfo/ocfs2-users > ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No Space left on the device.
Hi Aravind, Aravind Divakaran wrote: > Hi > > My ocfs filesystem has 270gb free space. > > FilesystemSize Used Avail Use% Mounted on > /dev/mapper/store 501G 232G 270G 47% /data > > INode details for ocfs filesystem is > > FilesystemInodes IUsed IFree IUse% Mounted on > /dev/mapper/store131225616 60588258 70637358 47% /data > > I have run the stat_sysdir.sh on /dev/mapper/store. The output is on the > below link. > > http://pastebin.com/4VLLCpRa I just went through the output and confirmed that the culprit is free space fragmentation. So please refer to http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189. Regards, Tao > > Please help me to resolve this issue. > > Rgds, > Aravind M D > > > >> Hi Aravind, >> >> Sounds like you might have hit the free space fragmentation issue: >> http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189 >> >> I'm sure that if you post output of stat_sysdir.sh >> (http://oss.oracle.com/~seeda/misc/stat_sysdir.sh) one of the ocfs2 devs >> will be able to confirm this. >> >> *If* it is this problem, removing some node slots will help. That is of >> course if you have more node slots that you need. I think 8 are created by >> default. >> >> Cheers, >> >> Brad >> >> >> On Thu, 4 Mar 2010 10:28:49 +0530 (IST) >> "Aravind Divakaran" wrote: >> >>> HiAll, >>> >>> For my mailserver i am using ocfs2 filesystem configured on san. Now my >>> mail delivery application is sometimes complaining No Space left on the >>> device, even though there is enough space and inodes. Can anyone help me >>> to solve this issue. >>> >>> >>> Rgds, >>> Aravind M D >>> >>> >>> >>> ___ >>> Ocfs2-users mailing list >>> Ocfs2-users@oss.oracle.com >>> http://oss.oracle.com/mailman/listinfo/ocfs2-users >> ___ >> Ocfs2-users mailing list >> Ocfs2-users@oss.oracle.com >> http://oss.oracle.com/mailman/listinfo/ocfs2-users > > > > > > ___ > Ocfs2-users mailing list > Ocfs2-users@oss.oracle.com > http://oss.oracle.com/mailman/listinfo/ocfs2-users ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No Space left on the device.
Hi My ocfs filesystem has 270gb free space. FilesystemSize Used Avail Use% Mounted on /dev/mapper/store 501G 232G 270G 47% /data INode details for ocfs filesystem is FilesystemInodes IUsed IFree IUse% Mounted on /dev/mapper/store131225616 60588258 70637358 47% /data I have run the stat_sysdir.sh on /dev/mapper/store. The output is on the below link. http://pastebin.com/4VLLCpRa Please help me to resolve this issue. Rgds, Aravind M D > Hi Aravind, > > Sounds like you might have hit the free space fragmentation issue: > http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189 > > I'm sure that if you post output of stat_sysdir.sh > (http://oss.oracle.com/~seeda/misc/stat_sysdir.sh) one of the ocfs2 devs > will be able to confirm this. > > *If* it is this problem, removing some node slots will help. That is of > course if you have more node slots that you need. I think 8 are created by > default. > > Cheers, > > Brad > > > On Thu, 4 Mar 2010 10:28:49 +0530 (IST) > "Aravind Divakaran" wrote: > >> HiAll, >> >> For my mailserver i am using ocfs2 filesystem configured on san. Now my >> mail delivery application is sometimes complaining No Space left on the >> device, even though there is enough space and inodes. Can anyone help me >> to solve this issue. >> >> >> Rgds, >> Aravind M D >> >> >> >> ___ >> Ocfs2-users mailing list >> Ocfs2-users@oss.oracle.com >> http://oss.oracle.com/mailman/listinfo/ocfs2-users > ___ > Ocfs2-users mailing list > Ocfs2-users@oss.oracle.com > http://oss.oracle.com/mailman/listinfo/ocfs2-users ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] No Space left on the device.
Hi Aravind, Sounds like you might have hit the free space fragmentation issue: http://oss.oracle.com/bugzilla/show_bug.cgi?id=1189 I'm sure that if you post output of stat_sysdir.sh (http://oss.oracle.com/~seeda/misc/stat_sysdir.sh) one of the ocfs2 devs will be able to confirm this. *If* it is this problem, removing some node slots will help. That is of course if you have more node slots that you need. I think 8 are created by default. Cheers, Brad On Thu, 4 Mar 2010 10:28:49 +0530 (IST) "Aravind Divakaran" wrote: > HiAll, > > For my mailserver i am using ocfs2 filesystem configured on san. Now my > mail delivery application is sometimes complaining No Space left on the > device, even though there is enough space and inodes. Can anyone help me > to solve this issue. > > > Rgds, > Aravind M D > > > > ___ > Ocfs2-users mailing list > Ocfs2-users@oss.oracle.com > http://oss.oracle.com/mailman/listinfo/ocfs2-users signature.asc Description: PGP signature ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
[Ocfs2-users] No Space left on the device.
HiAll, For my mailserver i am using ocfs2 filesystem configured on san. Now my mail delivery application is sometimes complaining No Space left on the device, even though there is enough space and inodes. Can anyone help me to solve this issue. Rgds, Aravind M D ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
