can anyone make it at server side? i don't know anything about server
side source code (yet)
2011/2/4 Adam Lee <a...@fotolog.biz>:
> awl
>
> On Feb 3, 2011 11:11 PM, "Roberto Spadim" <robe...@spadim.com.br> wrote:
>> not identical... if you need less network latency, server side is
>> better... no doubt...
>> like CAS we could make a LOCK system (just LOCK/UNLOCK, with timeout,
>> and just allow change lock/unlock if content of lock = content value
>> sent on memcache client)
>>
>> 2011/2/3 Adam Lee <a...@fotolog.biz>:
>>> I'm not sure what you mean that it's "client-based." Sure, the logic is
>>> on
>>> the client, as a lot of things are with memcached, but the CAS is
>>> enforced
>>> by the server. Doesn't seem like it's functionally any different than
>>> adding the same function on the server side, since the client would just
>>> be
>>> interpreting the new feature in the protocol instead, but functionally it
>>> would work out to be identical, I believe...
>>>
>>> On Thu, Feb 3, 2011 at 2:49 PM, Roberto Spadim <robe...@spadim.com.br>
>>> wrote:
>>>>
>>>> ok, but it´s client based...
>>>> i want a server based (memcache daemon)
>>>>
>>>> at client side:
>>>> memcached_lock("lock_name",1);
>>>> memcached_lock("lock_name",0);
>>>>
>>>> at server side:
>>>> lock/unlock some variable (maybe a server based flock())
>>>> since we use ram memory we could use ram locks (not filesystem lock)
>>>> with repcache we can replicate this lock on replicas....
>>>>
>>>>
>>>> 2011/2/3 Adam Lee <a...@fotolog.biz>:
>>>> > Here's one I hacked together a while back, though, as I said before, I
>>>> > recommend using something better suited to the job... BTW, this thing
>>>> > uses
>>>> > a few of our utility classes, but it should be very simple to drop in
>>>> > replacements.
>>>> > public class GlobalLock
>>>> > {
>>>> > public GlobalLock(String lockType, String resourceId)
>>>> > {
>>>> > if (StringUtils.isBlank(lockType))
>>>> > throw new NullPointerException("Empty lock type");
>>>> > if (StringUtils.isBlank(resourceId))
>>>> > throw new NullPointerException("Empty resource id");
>>>> > _globalId = StringUtil.toHexString((lockType +
>>>> > resourceId).getBytes()) +":GlobalLock";
>>>> > while(_value == 0)
>>>> > _value = RandomUtils.nextLong();
>>>> > _acquired = false;
>>>> > }
>>>> > public GlobalLock lock()
>>>> > {
>>>> > if (_acquired)
>>>> > return this;
>>>> > // Lock duration 20sec
>>>> > // tries to acquire lock for 21sec
>>>> > // obviously this is a far from perfect hack
>>>> > final int LOCK_DURATION_SECS = 20;
>>>> > final int SLEEP_TIME_MILLIS = 100;
>>>> > final int MAX_TRIES = 210; // max number of attempts to
>>>> > acquire
>>>> > lock
>>>> > MemcachedClient mc = FotologMemCache.getFotolog();
>>>> > for(int numTries = 0; numTries < MAX_TRIES; numTries++)
>>>> > {
>>>> > if (_log.isInfoEnabled()) _log.info("locking "+_globalId);
>>>> > try
>>>> > {
>>>> > CASValue<Object> mcVal = mc.gets(_globalId);
>>>> > if (mcVal == null)
>>>> > {
>>>> > _acquired = mc.add(_globalId, LOCK_DURATION_SECS,
>>>> > _value).get();
>>>> > }
>>>> > else if ( ((Long)mcVal.getValue()).longValue() == 0 )
>>>> > {
>>>> > CASResponse casResp = mc.cas(_globalId,
>>>> > mcVal.getCas(),
>>>> > _value);
>>>> > _acquired = (casResp == CASResponse.OK);
>>>> > }
>>>> > else
>>>> > {
>>>> > if (_log.isInfoEnabled()) _log.info("waiting for
>>>> > another
>>>> > process to finish: "+_globalId + ":" + mcVal.getValue());
>>>> > }
>>>> > }
>>>> > catch (Exception e)
>>>> > {
>>>> > _log.error(e.getMessage());
>>>> > }
>>>> > if (_acquired)
>>>> > return this;
>>>> > try { Thread.sleep(SLEEP_TIME_MILLIS); } catch
>>>> > (InterruptedException ie) {/**/} // don't 'busywait'
>>>> > }
>>>> > throw new GlobalLockException("Unable to lock [" + _globalId +
>>>> > "]
>>>> > after " + MAX_TRIES + " attempts");
>>>> > }
>>>> >
>>>> > /** Unlocks ALL of the resources locked with lock().
>>>> > * Never fails, so doesn't require additional try/catch if you are
>>>> > calling it from some other 'finally'
>>>> > */
>>>> > public void unlock()
>>>> > {
>>>> > if (_acquired)
>>>> > {
>>>> > _acquired = false;
>>>> > MemcachedClient mc = FotologMemCache.getFotolog();
>>>> > CASValue<Object> mcVal = mc.gets(_globalId);
>>>> > if (mcVal == null)
>>>> > {
>>>> > // nothing to do, val already expired
>>>> > if (_log.isInfoEnabled()) _log.info("already expired:
>>>> > "
>>>> > +
>>>> > _globalId + ":" + _value);
>>>> > return;
>>>> > }
>>>> > else if ( ((Long)mcVal.getValue()).longValue() == _value )
>>>> > {
>>>> > mc.cas(_globalId, mcVal.getCas(), 0l); // reset but
>>>> > only
>>>> > if
>>>> > it matches our val
>>>> > }
>>>> > else
>>>> > {
>>>> > _log.error("failed to unlock: " + _globalId + ":" +
>>>> > _value);
>>>> > }
>>>> > }
>>>> > }
>>>> >
>>>> > private static Logger _log = Logger.getLogger(GlobalLock.class);
>>>> > private String _globalId;
>>>> > private long _value;
>>>> > private boolean _acquired;
>>>> > }
>>>> > On Tue, Feb 1, 2011 at 1:40 PM, Roberto Spadim <robe...@spadim.com.br>
>>>> > wrote:
>>>> >>
>>>> >> LOCK should be something like this:
>>>> >>
>>>> >> <?php
>>>> >> // type=0 -> unlock
>>>> >> // type=1 -> lock
>>>> >> // client_name must change (use sessionID + username)
>>>> >> function
>>>> >>
>>>> >> memcache_flock($memcache_obj,$key,$type=0,$client_name='1',$timeout=0){
>>>> >>
>>>> >> $ret=memcache_add($memcache_obj,$key,$client_name,false,$timeout);
>>>> >> if($ret==true){
>>>> >> if($type==0) // delete
>>>> >> memcache_del($memcache_obj,$key);
>>>> >> return(true);
>>>> >> }
>>>> >> $cur_cli=memcache_get($memcache_obj,$key);
>>>> >> if(is_string($cur_cli) && $cur_cli!=''){ // if ='' no user!
>>>> >> if($cur_cli !== $client_name){
>>>> >> // it's not our lock
>>>> >> if(check_user_online_function()) // http
>>>> >> session
>>>> >> function (if want
>>>> >> http session integration), for memcached it´s like (true)
>>>> >> return($cur_cli); // return
>>>> >> current
>>>> >> lock client_name
>>>> >> }
>>>> >> // our lock!
>>>> >> }else{
>>>> >> // replace, autocorrect a wrong usage
>>>> >> memcache_replace($memcache_obj, $key, $client_name,
>>>> >> false,
>>>> >> $timeout);
>>>> >> }
>>>> >> if($type==0) // delete?
>>>> >> memcache_del($memcache_obj,$key);
>>>> >> return(true);
>>>> >> }
>>>> >> ?>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >> 2011/2/1 Adam Lee <a...@fotolog.biz>:
>>>> >> > there are some excellent solutions out there already. check out,
>>>> >> > for
>>>> >> > example, zookeeper.
>>>> >> >
>>>> >> > awl
>>>> >> >
>>>> >> > On Jan 29, 2011 3:32 PM, "rspadim" <rspa...@gmail.com> wrote:
>>>> >> >> hi guys, there's a async replication project (repcached) that is
>>>> >> >> very
>>>> >> >> interesting, could we implement it in main source code? at compile
>>>> >> >> time we could select from repcached or memcached
>>>> >> >> could we make it sync and/or async?
>>>> >> >> http://repcached.sourceforge.net/
>>>> >> >>
>>>> >> >>
>>>> >> >>
>>>> >> >>
>>>> >> >> =================================================================================
>>>> >> >> there's some non volatile solutions too that's very interesting
>>>> >> >> (memcachedb), for low memory computers we can use disk
>>>> >> >> could we implement it in main source code too?
>>>> >> >> http://memcachedb.org/
>>>> >> >>
>>>> >> >>
>>>> >> >>
>>>> >> >>
>>>> >> >>
>>>> >> >> =================================================================================
>>>> >> >> another, now !NEW! feature...
>>>> >> >>
>>>> >> >> i was looking for a *DISTRIBUTED LOCK MANAGER*, but i only found
>>>> >> >> kernel linux lock manager, that's based on file system (flock)
>>>> >> >> could we implement a lock manager at memcached?
>>>> >> >>
>>>> >> >> what lock manager do?
>>>> >> >> client send: KEY NAME, lock type+client name (KEY VALUE), key
>>>> >> >> timeout,
>>>> >> >> wait lock timeout (infinity/seconds)
>>>> >> >> (this can be implement in memcached protocol without many
>>>> >> >> modifications!!!)
>>>> >> >> server side function:
>>>> >> >> 1)seek if client can have this lock
>>>> >> >> 2)wait lock timeout... (this is a problem since we can have a very
>>>> >> >> big
>>>> >> >> wait time...)
>>>> >> >> 3) if client disconect exit do while
>>>> >> >> 4) yes we have the lock => change key value (give this lock to
>>>> >> >> client), exit do
>>>> >> >> 5) no we don't have the lock, exit do
>>>> >> >> 6) end of do while... return key value: lock type + client name
>>>> >> >> (like
>>>> >> >> a get command)
>>>> >> >>
>>>> >> >> ideas:
>>>> >> >> 1)maybe a separated memory size? we can run two separated servers,
>>>> >> >> one
>>>> >> >> for keys another for lock function (make command line options:
>>>> >> >> just
>>>> >> >> lock system, objects only system or both)
>>>> >> >>
>>>> >> >> 2)this type of key is diferent from memcached key cache objects,
>>>> >> >> that's obvious
>>>> >> >>
>>>> >> >> but........ is managed with same functions... (get, list, etc)
>>>> >> >> but........
>>>> >> >> all write/delete functions can't be done, they MUST be done by
>>>> >> >> LOCK
>>>> >> >> (the new) function,
>>>> >> >> DELETE/UNLOCK function is a LOCK function with lock type=0
>>>> >> >> (unlock)
>>>> >> >> read can be done by get and will return current client lock name
>>>> >> >> and
>>>> >> >> lock type (get command)
>>>> >> >>
>>>> >> >>
>>>> >> >>
>>>> >> >> *WHY THIS FEATURE?*
>>>> >> >> i didn't found a distributed lock manager for user space (not
>>>> >> >> kernel
>>>> >> >> space) with easy to implement protocol, and many program
>>>> >> >> languages,
>>>> >> >> and a very mature server and protocol.
>>>> >> >> =(
>>>> >> >>
>>>> >> >> but with this feature...
>>>> >> >> I DON'T NEED A SAMBA/NFS SERVER FOR NON FILESYSTEM LOCKING!!!!!
>>>> >> >> \o/
>>>> >> >> I WILL NEVER USE FLOCK() AGAIN!!! \o/ !!!
>>>> >> >>
>>>> >> >> I JUST NEED:
>>>> >> >> MYSQL+MEMCACHED+ (APACHE+CGI/PHP/JAVA/PERL/PYTHON)
>>>> >> >> for any cluster solution, no more filesystem!!!
>>>> >> >>
>>>> >> >> NO MORE FILESYSTEM REPLICATIONS (DRBD, NBD+RAID) FOR MY HIGH
>>>> >> >> AVAIBILITY / CLUSTER SOLUTION!!!!!
>>>> >> >> WE CAN USE REPCACHED (WE NEED A SYNC MODE)....
>>>> >> >>
>>>> >> >> THINK ABOUT IT!!!
>>>> >> >> REPLICATION + FLOCK!!!!! IT'S A VERY VERY VERY NICE FEATURE!!!!!
>>>> >> >>
>>>> >> >> ====================
>>>> >> >> type of object (1bit) default / lock manager can be putted on key
>>>> >> >> options/flags!!!
>>>> >> >> inside key value, we can put:
>>>> >> >> lock type(3 bits)
>>>> >> >> client name (a variable length, many bytes)
>>>> >> >>
>>>> >> >> http://en.wikipedia.org/wiki/Distributed_lock_manager
>>>> >> >> from wikipedia, TYPE OF LOCKS:
>>>> >> >> * Null Lock (NL). Indicates interest in the resource, but does not
>>>> >> >> prevent other processes from locking it. It has the advantage that
>>>> >> >> the
>>>> >> >> resource and its lock value block are preserved, even when no
>>>> >> >> processes are locking it.
>>>> >> >> * Concurrent Read (CR). Indicates a desire to read (but not
>>>> >> >> update) the resource. It allows other processes to read or update
>>>> >> >> the
>>>> >> >> resource, but prevents others from having exclusive access to it.
>>>> >> >> This
>>>> >> >> is usually employed on high-level resources, in order that more
>>>> >> >> restrictive locks can be obtained on subordinate resources.
>>>> >> >> * Concurrent Write (CW). Indicates a desire to read and update the
>>>> >> >> resource. It also allows other processes to read or update the
>>>> >> >> resource, but prevents others from having exclusive access to it.
>>>> >> >> This
>>>> >> >> is also usually employed on high-level resources, in order that
>>>> >> >> more
>>>> >> >> restrictive locks can be obtained on subordinate resources.
>>>> >> >> * Protected Read (PR). This is the traditional share lock, which
>>>> >> >> indicates a desire to read the resource but prevents other from
>>>> >> >> updating it. Others can however also read the resource.
>>>> >> >> * Protected Write (PW). This is the traditional update lock, which
>>>> >> >> indicates a desire to read and update the resource and prevents
>>>> >> >> others
>>>> >> >> from updating it. Others with Concurrent Read access can however
>>>> >> >> read
>>>> >> >> the resource.
>>>> >> >> * Exclusive (EX). This is the traditional exclusive lock which
>>>> >> >> allows read and update access to the resource, and prevents others
>>>> >> >> from having any access to it.
>>>> >> >>
>>>> >> >> NEW LOCK FUNCTION:
>>>> >> >>
>>>> >> >> LOCK <key><lock_type><client name><timeout><wait lock timeout>
>>>> >> >>
>>>> >> >> key: key name
>>>> >> >> <lock_type+client_name>=key value
>>>> >> >>
>>>> >> >> lock_type:
>>>> >> >> NL = 0
>>>> >> >> CR = 1
>>>> >> >> CW = 2
>>>> >> >> PR = 3
>>>> >> >> PW = 4
>>>> >> >> EX = 5
>>>> >> >>
>>>> >> >> client name: any value
>>>> >> >> timeout: any number, 0=infinity
>>>> >> >> wait lock timeout: wait lock time, 0=infinity
>>>> >> >>
>>>> >> >> how lock works: (see that lock type is only 0 or !=0 in this
>>>> >> >> logic...)
>>>> >> >> i will use <sent xxxx> for user new value, and <current xxx> for
>>>> >> >> the
>>>> >> >> current server value
>>>> >> >>
>>>> >> >> if key don't exists, create
>>>> >> >> do{
>>>> >> >> if ((sent_lock_type = 0 and sent_client_name =
>>>> >> >> current_client_name)
>>>> >> >> or key_timed_out==1)
>>>> >> >> remove key (delete)
>>>> >> >> send null lock and sent_client_name information
>>>> >> >> exit function
>>>> >> >> }else if (sent lock type = (1 or 2 or 3 or 4 or 5), and current
>>>> >> >> user
>>>> >> >> = sent user)
>>>> >> >> set
>>>> >> >>
>>>> >> >>
>>>> >> >> current_client_name,current_lock_type=sent_client_name,sent_lock_type
>>>> >> >> exit do
>>>> >> >> }else{
>>>> >> >> if wait lock time < time waiting lock to occur
>>>> >> >> exit do
>>>> >> >> }
>>>> >> >> }while(1)
>>>> >> >> send current_lock_type and sent_client_name
>>>> >> >> exit function
>>>> >> >>
>>>> >> >>
>>>> >> >> thanks guys!!!
>>>> >> >
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Roberto Spadim
>>>> >> Spadim Technology / SPAEmpresarial
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > awl
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Roberto Spadim
>>>> Spadim Technology / SPAEmpresarial
>>>
>>>
>>>
>>> --
>>> awl
>>>
>>
>>
>>
>> --
>> Roberto Spadim
>> Spadim Technology / SPAEmpresarial
>
--
Roberto Spadim
Spadim Technology / SPAEmpresarial
