awl
On Feb 3, 2011 11:11 PM, "Roberto Spadim" <robe...@spadim.com.br> wrote:
> not identical... if you need less network latency, server side is
> better... no doubt...
> like CAS we could make a LOCK system (just LOCK/UNLOCK, with timeout,
> and just allow change lock/unlock if content of lock = content value
> sent on memcache client)
>
> 2011/2/3 Adam Lee <a...@fotolog.biz>:
>> I'm not sure what you mean that it's "client-based." Sure, the logic is
on
>> the client, as a lot of things are with memcached, but the CAS is
enforced
>> by the server. Doesn't seem like it's functionally any different than
>> adding the same function on the server side, since the client would just
be
>> interpreting the new feature in the protocol instead, but functionally it
>> would work out to be identical, I believe...
>>
>> On Thu, Feb 3, 2011 at 2:49 PM, Roberto Spadim <robe...@spadim.com.br>
>> wrote:
>>>
>>> ok, but it´s client based...
>>> i want a server based (memcache daemon)
>>>
>>> at client side:
>>> memcached_lock("lock_name",1);
>>> memcached_lock("lock_name",0);
>>>
>>> at server side:
>>> lock/unlock some variable (maybe a server based flock())
>>> since we use ram memory we could use ram locks (not filesystem lock)
>>> with repcache we can replicate this lock on replicas....
>>>
>>>
>>> 2011/2/3 Adam Lee <a...@fotolog.biz>:
>>> > Here's one I hacked together a while back, though, as I said before, I
>>> > recommend using something better suited to the job... BTW, this thing
>>> > uses
>>> > a few of our utility classes, but it should be very simple to drop in
>>> > replacements.
>>> > public class GlobalLock
>>> > {
>>> > public GlobalLock(String lockType, String resourceId)
>>> > {
>>> > if (StringUtils.isBlank(lockType))
>>> > throw new NullPointerException("Empty lock type");
>>> > if (StringUtils.isBlank(resourceId))
>>> > throw new NullPointerException("Empty resource id");
>>> > _globalId = StringUtil.toHexString((lockType +
>>> > resourceId).getBytes()) +":GlobalLock";
>>> > while(_value == 0)
>>> > _value = RandomUtils.nextLong();
>>> > _acquired = false;
>>> > }
>>> > public GlobalLock lock()
>>> > {
>>> > if (_acquired)
>>> > return this;
>>> > // Lock duration 20sec
>>> > // tries to acquire lock for 21sec
>>> > // obviously this is a far from perfect hack
>>> > final int LOCK_DURATION_SECS = 20;
>>> > final int SLEEP_TIME_MILLIS = 100;
>>> > final int MAX_TRIES = 210; // max number of attempts to
acquire
>>> > lock
>>> > MemcachedClient mc = FotologMemCache.getFotolog();
>>> > for(int numTries = 0; numTries < MAX_TRIES; numTries++)
>>> > {
>>> > if (_log.isInfoEnabled()) _log.info("locking "+_globalId);
>>> > try
>>> > {
>>> > CASValue<Object> mcVal = mc.gets(_globalId);
>>> > if (mcVal == null)
>>> > {
>>> > _acquired = mc.add(_globalId, LOCK_DURATION_SECS,
>>> > _value).get();
>>> > }
>>> > else if ( ((Long)mcVal.getValue()).longValue() == 0 )
>>> > {
>>> > CASResponse casResp = mc.cas(_globalId,
>>> > mcVal.getCas(),
>>> > _value);
>>> > _acquired = (casResp == CASResponse.OK);
>>> > }
>>> > else
>>> > {
>>> > if (_log.isInfoEnabled()) _log.info("waiting for
>>> > another
>>> > process to finish: "+_globalId + ":" + mcVal.getValue());
>>> > }
>>> > }
>>> > catch (Exception e)
>>> > {
>>> > _log.error(e.getMessage());
>>> > }
>>> > if (_acquired)
>>> > return this;
>>> > try { Thread.sleep(SLEEP_TIME_MILLIS); } catch
>>> > (InterruptedException ie) {/**/} // don't 'busywait'
>>> > }
>>> > throw new GlobalLockException("Unable to lock [" + _globalId +
>>> > "]
>>> > after " + MAX_TRIES + " attempts");
>>> > }
>>> >
>>> > /** Unlocks ALL of the resources locked with lock().
>>> > * Never fails, so doesn't require additional try/catch if you are
>>> > calling it from some other 'finally'
>>> > */
>>> > public void unlock()
>>> > {
>>> > if (_acquired)
>>> > {
>>> > _acquired = false;
>>> > MemcachedClient mc = FotologMemCache.getFotolog();
>>> > CASValue<Object> mcVal = mc.gets(_globalId);
>>> > if (mcVal == null)
>>> > {
>>> > // nothing to do, val already expired
>>> > if (_log.isInfoEnabled()) _log.info("already expired:
"
>>> > +
>>> > _globalId + ":" + _value);
>>> > return;
>>> > }
>>> > else if ( ((Long)mcVal.getValue()).longValue() == _value )
>>> > {
>>> > mc.cas(_globalId, mcVal.getCas(), 0l); // reset but
only
>>> > if
>>> > it matches our val
>>> > }
>>> > else
>>> > {
>>> > _log.error("failed to unlock: " + _globalId + ":" +
>>> > _value);
>>> > }
>>> > }
>>> > }
>>> >
>>> > private static Logger _log = Logger.getLogger(GlobalLock.class);
>>> > private String _globalId;
>>> > private long _value;
>>> > private boolean _acquired;
>>> > }
>>> > On Tue, Feb 1, 2011 at 1:40 PM, Roberto Spadim <robe...@spadim.com.br>
>>> > wrote:
>>> >>
>>> >> LOCK should be something like this:
>>> >>
>>> >> <?php
>>> >> // type=0 -> unlock
>>> >> // type=1 -> lock
>>> >> // client_name must change (use sessionID + username)
>>> >> function
>>> >>
memcache_flock($memcache_obj,$key,$type=0,$client_name='1',$timeout=0){
>>> >>
>>> >> $ret=memcache_add($memcache_obj,$key,$client_name,false,$timeout);
>>> >> if($ret==true){
>>> >> if($type==0) // delete
>>> >> memcache_del($memcache_obj,$key);
>>> >> return(true);
>>> >> }
>>> >> $cur_cli=memcache_get($memcache_obj,$key);
>>> >> if(is_string($cur_cli) && $cur_cli!=''){ // if ='' no user!
>>> >> if($cur_cli !== $client_name){
>>> >> // it's not our lock
>>> >> if(check_user_online_function()) // http
session
>>> >> function (if want
>>> >> http session integration), for memcached it´s like (true)
>>> >> return($cur_cli); // return
>>> >> current
>>> >> lock client_name
>>> >> }
>>> >> // our lock!
>>> >> }else{
>>> >> // replace, autocorrect a wrong usage
>>> >> memcache_replace($memcache_obj, $key, $client_name,
>>> >> false,
>>> >> $timeout);
>>> >> }
>>> >> if($type==0) // delete?
>>> >> memcache_del($memcache_obj,$key);
>>> >> return(true);
>>> >> }
>>> >> ?>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> 2011/2/1 Adam Lee <a...@fotolog.biz>:
>>> >> > there are some excellent solutions out there already. check out,
for
>>> >> > example, zookeeper.
>>> >> >
>>> >> > awl
>>> >> >
>>> >> > On Jan 29, 2011 3:32 PM, "rspadim" <rspa...@gmail.com> wrote:
>>> >> >> hi guys, there's a async replication project (repcached) that is
>>> >> >> very
>>> >> >> interesting, could we implement it in main source code? at compile
>>> >> >> time we could select from repcached or memcached
>>> >> >> could we make it sync and/or async?
>>> >> >> http://repcached.sourceforge.net/
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >>
=================================================================================
>>> >> >> there's some non volatile solutions too that's very interesting
>>> >> >> (memcachedb), for low memory computers we can use disk
>>> >> >> could we implement it in main source code too?
>>> >> >> http://memcachedb.org/
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >>
=================================================================================
>>> >> >> another, now !NEW! feature...
>>> >> >>
>>> >> >> i was looking for a *DISTRIBUTED LOCK MANAGER*, but i only found
>>> >> >> kernel linux lock manager, that's based on file system (flock)
>>> >> >> could we implement a lock manager at memcached?
>>> >> >>
>>> >> >> what lock manager do?
>>> >> >> client send: KEY NAME, lock type+client name (KEY VALUE), key
>>> >> >> timeout,
>>> >> >> wait lock timeout (infinity/seconds)
>>> >> >> (this can be implement in memcached protocol without many
>>> >> >> modifications!!!)
>>> >> >> server side function:
>>> >> >> 1)seek if client can have this lock
>>> >> >> 2)wait lock timeout... (this is a problem since we can have a very
>>> >> >> big
>>> >> >> wait time...)
>>> >> >> 3) if client disconect exit do while
>>> >> >> 4) yes we have the lock => change key value (give this lock to
>>> >> >> client), exit do
>>> >> >> 5) no we don't have the lock, exit do
>>> >> >> 6) end of do while... return key value: lock type + client name
>>> >> >> (like
>>> >> >> a get command)
>>> >> >>
>>> >> >> ideas:
>>> >> >> 1)maybe a separated memory size? we can run two separated servers,
>>> >> >> one
>>> >> >> for keys another for lock function (make command line options:
just
>>> >> >> lock system, objects only system or both)
>>> >> >>
>>> >> >> 2)this type of key is diferent from memcached key cache objects,
>>> >> >> that's obvious
>>> >> >>
>>> >> >> but........ is managed with same functions... (get, list, etc)
>>> >> >> but........
>>> >> >> all write/delete functions can't be done, they MUST be done by
LOCK
>>> >> >> (the new) function,
>>> >> >> DELETE/UNLOCK function is a LOCK function with lock type=0
(unlock)
>>> >> >> read can be done by get and will return current client lock name
and
>>> >> >> lock type (get command)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> *WHY THIS FEATURE?*
>>> >> >> i didn't found a distributed lock manager for user space (not
kernel
>>> >> >> space) with easy to implement protocol, and many program
languages,
>>> >> >> and a very mature server and protocol.
>>> >> >> =(
>>> >> >>
>>> >> >> but with this feature...
>>> >> >> I DON'T NEED A SAMBA/NFS SERVER FOR NON FILESYSTEM LOCKING!!!!!
\o/
>>> >> >> I WILL NEVER USE FLOCK() AGAIN!!! \o/ !!!
>>> >> >>
>>> >> >> I JUST NEED:
>>> >> >> MYSQL+MEMCACHED+ (APACHE+CGI/PHP/JAVA/PERL/PYTHON)
>>> >> >> for any cluster solution, no more filesystem!!!
>>> >> >>
>>> >> >> NO MORE FILESYSTEM REPLICATIONS (DRBD, NBD+RAID) FOR MY HIGH
>>> >> >> AVAIBILITY / CLUSTER SOLUTION!!!!!
>>> >> >> WE CAN USE REPCACHED (WE NEED A SYNC MODE)....
>>> >> >>
>>> >> >> THINK ABOUT IT!!!
>>> >> >> REPLICATION + FLOCK!!!!! IT'S A VERY VERY VERY NICE FEATURE!!!!!
>>> >> >>
>>> >> >> ====================
>>> >> >> type of object (1bit) default / lock manager can be putted on key
>>> >> >> options/flags!!!
>>> >> >> inside key value, we can put:
>>> >> >> lock type(3 bits)
>>> >> >> client name (a variable length, many bytes)
>>> >> >>
>>> >> >> http://en.wikipedia.org/wiki/Distributed_lock_manager
>>> >> >> from wikipedia, TYPE OF LOCKS:
>>> >> >> * Null Lock (NL). Indicates interest in the resource, but does not
>>> >> >> prevent other processes from locking it. It has the advantage that
>>> >> >> the
>>> >> >> resource and its lock value block are preserved, even when no
>>> >> >> processes are locking it.
>>> >> >> * Concurrent Read (CR). Indicates a desire to read (but not
>>> >> >> update) the resource. It allows other processes to read or update
>>> >> >> the
>>> >> >> resource, but prevents others from having exclusive access to it.
>>> >> >> This
>>> >> >> is usually employed on high-level resources, in order that more
>>> >> >> restrictive locks can be obtained on subordinate resources.
>>> >> >> * Concurrent Write (CW). Indicates a desire to read and update the
>>> >> >> resource. It also allows other processes to read or update the
>>> >> >> resource, but prevents others from having exclusive access to it.
>>> >> >> This
>>> >> >> is also usually employed on high-level resources, in order that
more
>>> >> >> restrictive locks can be obtained on subordinate resources.
>>> >> >> * Protected Read (PR). This is the traditional share lock, which
>>> >> >> indicates a desire to read the resource but prevents other from
>>> >> >> updating it. Others can however also read the resource.
>>> >> >> * Protected Write (PW). This is the traditional update lock, which
>>> >> >> indicates a desire to read and update the resource and prevents
>>> >> >> others
>>> >> >> from updating it. Others with Concurrent Read access can however
>>> >> >> read
>>> >> >> the resource.
>>> >> >> * Exclusive (EX). This is the traditional exclusive lock which
>>> >> >> allows read and update access to the resource, and prevents others
>>> >> >> from having any access to it.
>>> >> >>
>>> >> >> NEW LOCK FUNCTION:
>>> >> >>
>>> >> >> LOCK <key><lock_type><client name><timeout><wait lock timeout>
>>> >> >>
>>> >> >> key: key name
>>> >> >> <lock_type+client_name>=key value
>>> >> >>
>>> >> >> lock_type:
>>> >> >> NL = 0
>>> >> >> CR = 1
>>> >> >> CW = 2
>>> >> >> PR = 3
>>> >> >> PW = 4
>>> >> >> EX = 5
>>> >> >>
>>> >> >> client name: any value
>>> >> >> timeout: any number, 0=infinity
>>> >> >> wait lock timeout: wait lock time, 0=infinity
>>> >> >>
>>> >> >> how lock works: (see that lock type is only 0 or !=0 in this
>>> >> >> logic...)
>>> >> >> i will use <sent xxxx> for user new value, and <current xxx> for
the
>>> >> >> current server value
>>> >> >>
>>> >> >> if key don't exists, create
>>> >> >> do{
>>> >> >> if ((sent_lock_type = 0 and sent_client_name =
current_client_name)
>>> >> >> or key_timed_out==1)
>>> >> >> remove key (delete)
>>> >> >> send null lock and sent_client_name information
>>> >> >> exit function
>>> >> >> }else if (sent lock type = (1 or 2 or 3 or 4 or 5), and current
user
>>> >> >> = sent user)
>>> >> >> set
>>> >> >>
>>> >> >>
current_client_name,current_lock_type=sent_client_name,sent_lock_type
>>> >> >> exit do
>>> >> >> }else{
>>> >> >> if wait lock time < time waiting lock to occur
>>> >> >> exit do
>>> >> >> }
>>> >> >> }while(1)
>>> >> >> send current_lock_type and sent_client_name
>>> >> >> exit function
>>> >> >>
>>> >> >>
>>> >> >> thanks guys!!!
>>> >> >
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Roberto Spadim
>>> >> Spadim Technology / SPAEmpresarial
>>> >
>>> >
>>> >
>>> > --
>>> > awl
>>> >
>>>
>>>
>>>
>>> --
>>> Roberto Spadim
>>> Spadim Technology / SPAEmpresarial
>>
>>
>>
>> --
>> awl
>>
>
>
>
> --
> Roberto Spadim
> Spadim Technology / SPAEmpresarial
