not identical... if you need less network latency, server side is better... no doubt... like CAS we could make a LOCK system (just LOCK/UNLOCK, with timeout, and just allow change lock/unlock if content of lock = content value sent on memcache client)
2011/2/3 Adam Lee <a...@fotolog.biz>: > I'm not sure what you mean that it's "client-based." Sure, the logic is on > the client, as a lot of things are with memcached, but the CAS is enforced > by the server. Doesn't seem like it's functionally any different than > adding the same function on the server side, since the client would just be > interpreting the new feature in the protocol instead, but functionally it > would work out to be identical, I believe... > > On Thu, Feb 3, 2011 at 2:49 PM, Roberto Spadim <robe...@spadim.com.br> > wrote: >> >> ok, but it´s client based... >> i want a server based (memcache daemon) >> >> at client side: >> memcached_lock("lock_name",1); >> memcached_lock("lock_name",0); >> >> at server side: >> lock/unlock some variable (maybe a server based flock()) >> since we use ram memory we could use ram locks (not filesystem lock) >> with repcache we can replicate this lock on replicas.... >> >> >> 2011/2/3 Adam Lee <a...@fotolog.biz>: >> > Here's one I hacked together a while back, though, as I said before, I >> > recommend using something better suited to the job... BTW, this thing >> > uses >> > a few of our utility classes, but it should be very simple to drop in >> > replacements. >> > public class GlobalLock >> > { >> > public GlobalLock(String lockType, String resourceId) >> > { >> > if (StringUtils.isBlank(lockType)) >> > throw new NullPointerException("Empty lock type"); >> > if (StringUtils.isBlank(resourceId)) >> > throw new NullPointerException("Empty resource id"); >> > _globalId = StringUtil.toHexString((lockType + >> > resourceId).getBytes()) +":GlobalLock"; >> > while(_value == 0) >> > _value = RandomUtils.nextLong(); >> > _acquired = false; >> > } >> > public GlobalLock lock() >> > { >> > if (_acquired) >> > return this; >> > // Lock duration 20sec >> > // tries to acquire lock for 21sec >> > // obviously this is a far from perfect hack >> > final int LOCK_DURATION_SECS = 20; >> > final int SLEEP_TIME_MILLIS = 100; >> > final int MAX_TRIES = 210; // max number of attempts to acquire >> > lock >> > MemcachedClient mc = FotologMemCache.getFotolog(); >> > for(int numTries = 0; numTries < MAX_TRIES; numTries++) >> > { >> > if (_log.isInfoEnabled()) _log.info("locking "+_globalId); >> > try >> > { >> > CASValue<Object> mcVal = mc.gets(_globalId); >> > if (mcVal == null) >> > { >> > _acquired = mc.add(_globalId, LOCK_DURATION_SECS, >> > _value).get(); >> > } >> > else if ( ((Long)mcVal.getValue()).longValue() == 0 ) >> > { >> > CASResponse casResp = mc.cas(_globalId, >> > mcVal.getCas(), >> > _value); >> > _acquired = (casResp == CASResponse.OK); >> > } >> > else >> > { >> > if (_log.isInfoEnabled()) _log.info("waiting for >> > another >> > process to finish: "+_globalId + ":" + mcVal.getValue()); >> > } >> > } >> > catch (Exception e) >> > { >> > _log.error(e.getMessage()); >> > } >> > if (_acquired) >> > return this; >> > try { Thread.sleep(SLEEP_TIME_MILLIS); } catch >> > (InterruptedException ie) {/**/} // don't 'busywait' >> > } >> > throw new GlobalLockException("Unable to lock [" + _globalId + >> > "] >> > after " + MAX_TRIES + " attempts"); >> > } >> > >> > /** Unlocks ALL of the resources locked with lock(). >> > * Never fails, so doesn't require additional try/catch if you are >> > calling it from some other 'finally' >> > */ >> > public void unlock() >> > { >> > if (_acquired) >> > { >> > _acquired = false; >> > MemcachedClient mc = FotologMemCache.getFotolog(); >> > CASValue<Object> mcVal = mc.gets(_globalId); >> > if (mcVal == null) >> > { >> > // nothing to do, val already expired >> > if (_log.isInfoEnabled()) _log.info("already expired: " >> > + >> > _globalId + ":" + _value); >> > return; >> > } >> > else if ( ((Long)mcVal.getValue()).longValue() == _value ) >> > { >> > mc.cas(_globalId, mcVal.getCas(), 0l); // reset but only >> > if >> > it matches our val >> > } >> > else >> > { >> > _log.error("failed to unlock: " + _globalId + ":" + >> > _value); >> > } >> > } >> > } >> > >> > private static Logger _log = Logger.getLogger(GlobalLock.class); >> > private String _globalId; >> > private long _value; >> > private boolean _acquired; >> > } >> > On Tue, Feb 1, 2011 at 1:40 PM, Roberto Spadim <robe...@spadim.com.br> >> > wrote: >> >> >> >> LOCK should be something like this: >> >> >> >> <?php >> >> // type=0 -> unlock >> >> // type=1 -> lock >> >> // client_name must change (use sessionID + username) >> >> function >> >> memcache_flock($memcache_obj,$key,$type=0,$client_name='1',$timeout=0){ >> >> >> >> $ret=memcache_add($memcache_obj,$key,$client_name,false,$timeout); >> >> if($ret==true){ >> >> if($type==0) // delete >> >> memcache_del($memcache_obj,$key); >> >> return(true); >> >> } >> >> $cur_cli=memcache_get($memcache_obj,$key); >> >> if(is_string($cur_cli) && $cur_cli!=''){ // if ='' no user! >> >> if($cur_cli !== $client_name){ >> >> // it's not our lock >> >> if(check_user_online_function()) // http session >> >> function (if want >> >> http session integration), for memcached it´s like (true) >> >> return($cur_cli); // return >> >> current >> >> lock client_name >> >> } >> >> // our lock! >> >> }else{ >> >> // replace, autocorrect a wrong usage >> >> memcache_replace($memcache_obj, $key, $client_name, >> >> false, >> >> $timeout); >> >> } >> >> if($type==0) // delete? >> >> memcache_del($memcache_obj,$key); >> >> return(true); >> >> } >> >> ?> >> >> >> >> >> >> >> >> >> >> 2011/2/1 Adam Lee <a...@fotolog.biz>: >> >> > there are some excellent solutions out there already. check out, for >> >> > example, zookeeper. >> >> > >> >> > awl >> >> > >> >> > On Jan 29, 2011 3:32 PM, "rspadim" <rspa...@gmail.com> wrote: >> >> >> hi guys, there's a async replication project (repcached) that is >> >> >> very >> >> >> interesting, could we implement it in main source code? at compile >> >> >> time we could select from repcached or memcached >> >> >> could we make it sync and/or async? >> >> >> http://repcached.sourceforge.net/ >> >> >> >> >> >> >> >> >> >> >> >> ================================================================================= >> >> >> there's some non volatile solutions too that's very interesting >> >> >> (memcachedb), for low memory computers we can use disk >> >> >> could we implement it in main source code too? >> >> >> http://memcachedb.org/ >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> ================================================================================= >> >> >> another, now !NEW! feature... >> >> >> >> >> >> i was looking for a *DISTRIBUTED LOCK MANAGER*, but i only found >> >> >> kernel linux lock manager, that's based on file system (flock) >> >> >> could we implement a lock manager at memcached? >> >> >> >> >> >> what lock manager do? >> >> >> client send: KEY NAME, lock type+client name (KEY VALUE), key >> >> >> timeout, >> >> >> wait lock timeout (infinity/seconds) >> >> >> (this can be implement in memcached protocol without many >> >> >> modifications!!!) >> >> >> server side function: >> >> >> 1)seek if client can have this lock >> >> >> 2)wait lock timeout... (this is a problem since we can have a very >> >> >> big >> >> >> wait time...) >> >> >> 3) if client disconect exit do while >> >> >> 4) yes we have the lock => change key value (give this lock to >> >> >> client), exit do >> >> >> 5) no we don't have the lock, exit do >> >> >> 6) end of do while... return key value: lock type + client name >> >> >> (like >> >> >> a get command) >> >> >> >> >> >> ideas: >> >> >> 1)maybe a separated memory size? we can run two separated servers, >> >> >> one >> >> >> for keys another for lock function (make command line options: just >> >> >> lock system, objects only system or both) >> >> >> >> >> >> 2)this type of key is diferent from memcached key cache objects, >> >> >> that's obvious >> >> >> >> >> >> but........ is managed with same functions... (get, list, etc) >> >> >> but........ >> >> >> all write/delete functions can't be done, they MUST be done by LOCK >> >> >> (the new) function, >> >> >> DELETE/UNLOCK function is a LOCK function with lock type=0 (unlock) >> >> >> read can be done by get and will return current client lock name and >> >> >> lock type (get command) >> >> >> >> >> >> >> >> >> >> >> >> *WHY THIS FEATURE?* >> >> >> i didn't found a distributed lock manager for user space (not kernel >> >> >> space) with easy to implement protocol, and many program languages, >> >> >> and a very mature server and protocol. >> >> >> =( >> >> >> >> >> >> but with this feature... >> >> >> I DON'T NEED A SAMBA/NFS SERVER FOR NON FILESYSTEM LOCKING!!!!! \o/ >> >> >> I WILL NEVER USE FLOCK() AGAIN!!! \o/ !!! >> >> >> >> >> >> I JUST NEED: >> >> >> MYSQL+MEMCACHED+ (APACHE+CGI/PHP/JAVA/PERL/PYTHON) >> >> >> for any cluster solution, no more filesystem!!! >> >> >> >> >> >> NO MORE FILESYSTEM REPLICATIONS (DRBD, NBD+RAID) FOR MY HIGH >> >> >> AVAIBILITY / CLUSTER SOLUTION!!!!! >> >> >> WE CAN USE REPCACHED (WE NEED A SYNC MODE).... >> >> >> >> >> >> THINK ABOUT IT!!! >> >> >> REPLICATION + FLOCK!!!!! IT'S A VERY VERY VERY NICE FEATURE!!!!! >> >> >> >> >> >> ==================== >> >> >> type of object (1bit) default / lock manager can be putted on key >> >> >> options/flags!!! >> >> >> inside key value, we can put: >> >> >> lock type(3 bits) >> >> >> client name (a variable length, many bytes) >> >> >> >> >> >> http://en.wikipedia.org/wiki/Distributed_lock_manager >> >> >> from wikipedia, TYPE OF LOCKS: >> >> >> * Null Lock (NL). Indicates interest in the resource, but does not >> >> >> prevent other processes from locking it. It has the advantage that >> >> >> the >> >> >> resource and its lock value block are preserved, even when no >> >> >> processes are locking it. >> >> >> * Concurrent Read (CR). Indicates a desire to read (but not >> >> >> update) the resource. It allows other processes to read or update >> >> >> the >> >> >> resource, but prevents others from having exclusive access to it. >> >> >> This >> >> >> is usually employed on high-level resources, in order that more >> >> >> restrictive locks can be obtained on subordinate resources. >> >> >> * Concurrent Write (CW). Indicates a desire to read and update the >> >> >> resource. It also allows other processes to read or update the >> >> >> resource, but prevents others from having exclusive access to it. >> >> >> This >> >> >> is also usually employed on high-level resources, in order that more >> >> >> restrictive locks can be obtained on subordinate resources. >> >> >> * Protected Read (PR). This is the traditional share lock, which >> >> >> indicates a desire to read the resource but prevents other from >> >> >> updating it. Others can however also read the resource. >> >> >> * Protected Write (PW). This is the traditional update lock, which >> >> >> indicates a desire to read and update the resource and prevents >> >> >> others >> >> >> from updating it. Others with Concurrent Read access can however >> >> >> read >> >> >> the resource. >> >> >> * Exclusive (EX). This is the traditional exclusive lock which >> >> >> allows read and update access to the resource, and prevents others >> >> >> from having any access to it. >> >> >> >> >> >> NEW LOCK FUNCTION: >> >> >> >> >> >> LOCK <key><lock_type><client name><timeout><wait lock timeout> >> >> >> >> >> >> key: key name >> >> >> <lock_type+client_name>=key value >> >> >> >> >> >> lock_type: >> >> >> NL = 0 >> >> >> CR = 1 >> >> >> CW = 2 >> >> >> PR = 3 >> >> >> PW = 4 >> >> >> EX = 5 >> >> >> >> >> >> client name: any value >> >> >> timeout: any number, 0=infinity >> >> >> wait lock timeout: wait lock time, 0=infinity >> >> >> >> >> >> how lock works: (see that lock type is only 0 or !=0 in this >> >> >> logic...) >> >> >> i will use <sent xxxx> for user new value, and <current xxx> for the >> >> >> current server value >> >> >> >> >> >> if key don't exists, create >> >> >> do{ >> >> >> if ((sent_lock_type = 0 and sent_client_name = current_client_name) >> >> >> or key_timed_out==1) >> >> >> remove key (delete) >> >> >> send null lock and sent_client_name information >> >> >> exit function >> >> >> }else if (sent lock type = (1 or 2 or 3 or 4 or 5), and current user >> >> >> = sent user) >> >> >> set >> >> >> >> >> >> current_client_name,current_lock_type=sent_client_name,sent_lock_type >> >> >> exit do >> >> >> }else{ >> >> >> if wait lock time < time waiting lock to occur >> >> >> exit do >> >> >> } >> >> >> }while(1) >> >> >> send current_lock_type and sent_client_name >> >> >> exit function >> >> >> >> >> >> >> >> >> thanks guys!!! >> >> > >> >> >> >> >> >> >> >> -- >> >> Roberto Spadim >> >> Spadim Technology / SPAEmpresarial >> > >> > >> > >> > -- >> > awl >> > >> >> >> >> -- >> Roberto Spadim >> Spadim Technology / SPAEmpresarial > > > > -- > awl > -- Roberto Spadim Spadim Technology / SPAEmpresarial