Transferring this back the ceph-users. Sorry, I can't help with rbd issues. One thing I will say is that if you are mounting an rbd device with a filesystem on a machine to export ftp, you can't also export the same device via iSCSI.
David Zafman Senior Developer http://www.inktank.com On Aug 19, 2013, at 8:39 PM, PJ <linalin1...@gmail.com> wrote: > 2013/8/14 David Zafman <david.zaf...@inktank.com> > > On Aug 12, 2013, at 7:41 PM, Josh Durgin <josh.dur...@inktank.com> wrote: > > > On 08/12/2013 07:18 PM, PJ wrote: > >> > >> If the target rbd device only map on one virtual machine, format it as > >> ext4 and mount to two places > >> mount /dev/rbd0 /nfs --> for nfs server usage > >> mount /dev/rbd0 /ftp --> for ftp server usage > >> nfs and ftp servers run on the same virtual machine. Will file system > >> (ext4) help to handle the simultaneous access from nfs and ftp? > > > > I doubt that'll work perfectly on a normal disk, although rbd should > > behave the same in this case. Consider what happens when to be some > > issues when the same files are modified at once by the ftp and nfs > > servers. You could run ftp on an nfs client on a different machine > > safely. > > > > > Modern Linux kernels will do a bind mount when a block device is mounted on 2 > different directories. Think directory hard links. Simultaneous access > will NOT corrupt ext4, but as Josh said modifying the same file at once by > ftp and nfs isn't going produce good results. With file locking 2 nfs > clients could coordinate using advisory locking. > > David Zafman > Senior Developer > http://www.inktank.com > > > The first issue is reproduced, but there are changes to system configuration. > Due to hardware shortage, we only have one physical machine installed one OSD > and runs 6 virtual machines on it. Only one monitor (wistor-003) and one FTP > server (wistor-004), the other virtual machines are iSCSI servers. > > The log size is big because when enable FTP service for a rbd device, we have > a rbd map retry loop in case it fails (retry rbd map every 10 sec and last > for 3 minutes). Please download monitor log from below link, > https://www.dropbox.com/s/88cb9q91cjszuug/ceph-mon.wistor-003.log.zip > > Here are the operation steps: > 1. The pool rex is created > Around 2013-08-20 09:16:38~09:16:39 > 2. The first time to map rbd device on wistor-004 and it fails (all retries > failed) > Around 2013-08-20 09:17:43~09:20:46 (180 sec) > 3. Tried second time and it works, but still have 9 fails in retry loop > Around 2013-08-20 09:20:48~09:22:10 (82 sec) > > > >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
