Re: [Ocfs2-users] ping_pong using o2cb and cman
ping_long tests the fcntl() user locks. ocfs2 supports clustered fcntl() locking with cman and pacemaker cluster stacks. Not with o2cb. ocfs2 supports clustered flock() with all stacks. o2cb, cman and pacemaker. On 01/17/2011 07:25 AM, Dan Warner wrote: I was testing ocfs2 on a 2 node cluster set up. ocfs2-tools version is 1.6.3 ocfs2 kernel version is 2.6.36 Using cman on 2 nodes node02 dw # ping_pong -rwm /data/test.dat 3 data increment = 2 14 locks/sec node01 dw # ping_pong -rw /data/test.dat 3 data increment = 2 10 locks/sec node02 dw # ping_pong -r /data/test.dat 3 1980 locks/sec Using cman on 1 node node02 dw # ping_pong -rwm /data/test.dat 3 data increment = 1 3165 locks/sec node02 dw # ping_pong -rw /data/test.dat 3 data increment = 1 3289 locks/sec node02 dw # ping_pong -r /data/test.dat 3 3267 locks/sec The above seems reasonable and behaved as expected when the program was stopped and started repeatedly on either node. So it passed the test. Unfortunately o2cb didnt behave as expected. Using o2cb on 2 nodes node02 dw # ping_pong -rwm /data/test.dat 3 data increment = 199 data increment = 76 data increment = 1 data increment = 117 data increment = 84 data increment = 1 data increment = 212 data increment = 211 data increment = 1 data increment = 225 data increment = 85 data increment = 1 data increment = 237 data increment = 158 data increment = 1 data increment = 75 data increment = 235 data increment = 1 data increment = 199 data increment = 4 data increment = 1 171803 locks/sec and occasionally you will get the data increment remaining at 1 when 2 nodes have joined node02 dw # ping_pong -rw /data/test.dat 3 data increment = 2 45 locks/sec node01 dw # ping_pong -r /data/test.dat 3 2155175 locks/sec Using cman on 1 node node02 dw # ping_pong -rwm /data/test.dat 3 data increment = 1 2230553 locks/sec node02 dw # ping_pong -rw /data/test.dat 3 data increment = 1 299492 locks/sec node02 dw # ping_pong -r /data/test.dat 3 2249787 locks/sec The mmap test failed. The read/write test 'usually' worked but frequently didn't increment the 'data increment' to 2, when the program was running on both nodes. Any suggestions why this isn't working as expected? Thanks, Dan ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] ping_pong using o2cb and cman
OK thanks for the clarification. Is the support of fcntl() locking within o2cb, on the wishlist of features to be implemented at a future date? Is it even a priority to do? Thanks, Dan On 18 January 2011 18:00, Sunil Mushran sunil.mush...@oracle.com wrote: ping_long tests the fcntl() user locks. ocfs2 supports clustered fcntl() locking with cman and pacemaker cluster stacks. Not with o2cb. ocfs2 supports clustered flock() with all stacks. o2cb, cman and pacemaker. ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] ping_pong using o2cb and cman
fcntl() locking does not pair at all with dlm locking. So both ocfs2 and gfs2 use a special scheme just for this lock. Both use the ordered messaging provided by the corosync cluster engine. This lock is fully synchronised and thus is not as performant as it can be. (Both cman and pacemaker use cce.) But that's ok because fcntl()'s locking semantics is not what you think it is. The lock is almost useless in this day and age. The only use is supporting legacy apps. For more refer to Jeremy Allison's overview of this user lock. http://www.samba.org/samba/news/articles/low_point/tale_two_stds_os2.html Given that and considering most apps allow users to choose between fcntl() and flock(), we've decided for now to just support the latter with o2cb. flock() pairs well with dlm locking. On 01/18/2011 10:22 AM, Dan Warner wrote: OK thanks for the clarification. Is the support of fcntl() locking within o2cb, on the wishlist of features to be implemented at a future date? Is it even a priority to do? Thanks, Dan On 18 January 2011 18:00, Sunil Mushran sunil.mush...@oracle.com mailto:sunil.mush...@oracle.com wrote: ping_long tests the fcntl() user locks. ocfs2 supports clustered fcntl() locking with cman and pacemaker cluster stacks. Not with o2cb. ocfs2 supports clustered flock() with all stacks. o2cb, cman and pacemaker. ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
Re: [Ocfs2-users] ping_pong using o2cb and cman
OK great, that made it clear. I didn't know that locking was a bit of a can of worms after reading a few pages on in. I did very quickly modify the ping_pong code to change the locking mechanism to flock and the o2cb stack locks as expected producing slightly better performance than cman. From http://wiki.samba.org/index.php/CTDB_Setup OCFS2 1.4 offers cluster-wide byte-range locking, but the OCFS2 1.2 filesystem will need special handling, as it does not support cluster wide byte range locking. To work around this you will need to use some other sort of shared filesystem (such as NFS) for the private directory in Samba and the CTDB_RECOVERY_LOCK file in CTDB. So I take it that to use samba ocfs2 you need to use cman or pacemaker as they support byte range locking or as suggested above use NFS for certain directories and the o2cb stack Thanks for your time, Dan On 18 January 2011 19:06, Sunil Mushran sunil.mush...@oracle.com wrote: fcntl() locking does not pair at all with dlm locking. So both ocfs2 and gfs2 use a special scheme just for this lock. Both use the ordered messaging provided by the corosync cluster engine. This lock is fully synchronised and thus is not as performant as it can be. (Both cman and pacemaker use cce.) But that's ok because fcntl()'s locking semantics is not what you think it is. The lock is almost useless in this day and age. The only use is supporting legacy apps. For more refer to Jeremy Allison's overview of this user lock. http://www.samba.org/samba/news/articles/low_point/tale_two_stds_os2.html Given that and considering most apps allow users to choose between fcntl() and flock(), we've decided for now to just support the latter with o2cb. flock() pairs well with dlm locking. On 01/18/2011 10:22 AM, Dan Warner wrote: OK thanks for the clarification. Is the support of fcntl() locking within o2cb, on the wishlist of features to be implemented at a future date? Is it even a priority to do? Thanks, Dan On 18 January 2011 18:00, Sunil Mushran sunil.mush...@oracle.com wrote: ping_long tests the fcntl() user locks. ocfs2 supports clustered fcntl() locking with cman and pacemaker cluster stacks. Not with o2cb. ocfs2 supports clustered flock() with all stacks. o2cb, cman and pacemaker. ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.comhttp://oss.oracle.com/mailman/listinfo/ocfs2-users ___ Ocfs2-users mailing list Ocfs2-users@oss.oracle.com http://oss.oracle.com/mailman/listinfo/ocfs2-users
