> So where is new document patch? I'm sending new document patch. I described the way to install memcached or libmemcached(c client library) for memcached users.
regards Masanori YAMAZAKI 2011/6/19 Tatsuo Ishii <[email protected]> > Forgot to attach patches. > -- > Tatsuo Ishii > SRA OSS, Inc. Japan > English: http://www.sraoss.co.jp/index_en.php > Japanese: http://www.sraoss.co.jp > > >> 2011/6/14 Tatsuo Ishii <[email protected]> > >> > >>> Please explain what is implemented and what is not implemented in this > >>> patch. > >>> > >> > >> What is implemented in this patch are as below. > >> - connect to memcached. > >> - set RowDescription or DataRow on memcached. > >> - get RowDescription or DataRow on memcached by using key of md5(SELECT > >> convert). > >> - delete cache on memcached by using key of md5(SELECT convert). only > >> module. > >> - fetch and parse the fellowing data in pgpool.conf. > >> > >> memory_cache_enabled > >> memqcache_method > >> memqcache_memcached_host > >> memqcache_memcached_port > >> memqcache_total_size > >> memqcache_expire > >> memqcache_maxcache > >> memqcache_cache_block_size > >> > >> - add data to buffer before set data on memcached. > >> - get buffer in order to set it on memcached > >> - initialize buffer > >> > >> I am implemented only in Simple Query case. > >> > >> > >> What is not implemented in this patch are as below. > >> - shmem cache. > >> structure of managing table oids. > >> the space management map. > >> on memory query cache structure. > >> > >> - implementation in Extended Query case. > >> > >> - cache validation. > >> The result of SELECT using none IMMUTABLE functions is not cached. > > > > I have made a patch to implement a function which checks if a SELECT > > statement includes non IMMUTABLE function call. > > > > extern bool pool_has_non_immutable_function_call(Node *node); > > > > See attached patches. > > > > TODO: > > > > Some statements including: > > > > SELECT current_timestamp; > > > > is not detected by the function even if it uses IMMUTABLE function > > because it is translated to a non function form. Obviously we should > > not cache the result of current_timestamp. What should we do? Probably > > we have to have a heuristics to detect the particular form? > > > >> - remove cache > >> If cache is full, the least recently used cache is removed. > >> If a table is dropped or modified, related cache data is deleted. > >> If a table definition is modified(ALTER TABLE), related cache data is > >> deleted. > >> If a schema or database is dropped, related cache data is deleted. > >> > >> > >> This week I am mainly implementing the shmem cache. > >> > >> Regards > >> Masanori YAMAZAKI > >> > >> > >> > >> > >>> -- > >>> Tatsuo Ishii > >>> SRA OSS, Inc. Japan > >>> English: http://www.sraoss.co.jp/index_en.php > >>> Japanese: http://www.sraoss.co.jp > >>> > >>> > Hello, hackers > >>> > > >>> > I have implemented a function caching query on memcached. > >>> > I am sending the patch file and it's so nice to give me some > opinions. > >>> > > >>> > Main program which I implemented is as below. > >>> > - Search for cache data on memcached by using md5 which converted > SELECT > >>> > into. > >>> > - Set data on memcached. The key is md5 which converted SELECT into. > >>> > The value is RowDescription, DataRow from Backend. > >>> > > >>> > [Modify files] > >>> > - pool.h > >>> > - child.c > >>> > - pool_config.l > >>> > - pool_config.h > >>> > - pool_config.c > >>> > - pool_system.c > >>> > - pool_proto_modules.c > >>> > - pool_process_query.c > >>> > > >>> > [New File] > >>> > - pool_memqcache.c > >>> > > >>> > > >>> > -- > >>> > Regards > >>> > Masanori YAMAZAKI > >>> > -- best regards Masanori YAMAZAKI email: [email protected]
Index: pgpool-en.html =================================================================== RCS file: /cvsroot/pgpool/pgpool-II/doc/pgpool-en.html,v retrieving revision 1.91 diff -c -r1.91 pgpool-en.html *** pgpool-en.html 1 May 2011 09:41:48 -0000 1.91 --- pgpool-en.html 19 Jun 2011 14:36:01 -0000 *************** *** 2050,2057 **** </p> <h2>Setting Query cache method</h2> ! <p>The Query cache can be used in all modes in pgpool-II. Activating it in ! pgpool.conf is done as follows:</p> <pre> enable_query_cache = true </pre> --- 2050,2063 ---- </p> <h2>Setting Query cache method</h2> ! <p>The Query cache can be used in all modes in pgpool-II.</p> ! ! <h3>Using SystemDB</h3> ! ! <p> ! Activating it in pgpool.conf is done as follows: ! </p> ! <pre> enable_query_cache = true </pre> *************** *** 2074,2079 **** --- 2080,2227 ---- "pgpool_catalog". </p> + <h3>Using Shared memory or Memcached</h3> + + <p> + To enable the memory cache functionality, set this to true (default is false) + </p> + + <pre> + memory_cache_enabled = true + </pre> + + <p> + Memory cache behavior can be specified by cache_param directive:<br> + Cache store method. Either "shmem"(shared memory) or "memcached". Default is shmem. + </p> + <pre> + memqcache_method = 'shmem' + </pre> + + <p> + Memcached host name. Mandatory if memqcache_method=memcached. If this parameter is starting with "/", UNIX domain socket is assumed. + </p> + <pre> + memqcache_memcached_host = 'foo.com' + </pre> + + <p> + Memcached port number. Mondatory if memqcache_method=memcached. + </p> + <pre> + memqcache_memcached_port = 11211 + </pre> + + <p> + Total memory size in bytes for storing memory cache. Mandatory if memqcache_method=shmem. + </p> + <pre> + memqcache_total_size = 10240 + </pre> + + <p> + Memory cache entry life time specified in seconds. Default is 60. + </p> + <pre> + memqcache_expire = 60 + </pre> + + <p> + Maximum SELECT result size in bytes. If exceeds this value, the result is not cached. + </p> + <pre> + memqcache_maxcache = 512 + </pre> + + <p> + Cache block size in bytes<br> + Only meaning full when memqcache_method=shmem. Memory cache is devided into fixed size of block.<br> + If the block size is small, memory usage is efficient, but processing might be slow. On the other hand,<br> + if the block size is large, memory usage is not efficient but processing might be fast. Default is 8192. <br> + + </p> + <pre> + memqcache_cache_block_size= 8192 + </pre> + + <h3>memcached Installation</h3> + + <p> + If you want to use memcached based query, you need to install memcached. + </p> + + <p> + memcached's source code can be downloaded from: + <a href="http://memcached.org/";>memcached development page</a> + </p> + + <dl> + <dt>configure</dt> + <dd> + <p> + After extracting the source tarball, execute the configure script. + </p> + <pre> + ./configure + </pre> + </dd> + <dt>make</dt> + <dd> + <pre> + make + make install + </pre> + </dd> + </dl> + + <h3>libmemcached Installation</h3> + + <p> + C client library used is libmemcached. + You need to install libmemcached after installing memcached. + </p> + + <p> + libmemcached's source code can be downloaded from: + <a href="http://libmemcached.org/libMemcached.html";>libmemcached development page</a> + </p> + + <dl> + <dt>configure</dt> + <dd> + <p> + After extracting the source tarball, execute the configure script. + </p> + <pre> + ./configure + </pre> + + <p> + If you want non-default values, some options can be set: + </p> + + <ul> + <li><code>--with-memcached=path</code><br/> + The top directory where Memcached are installed.</li> + </ul> + + </dd> + <dt>make</dt> + <dd> + <pre> + make + make install + </pre> + </dd> + </dl> + + + + + + + + <h1>Starting/Stopping pgpool-II<a name="start"></a></h1> <p>All the backends and the System DB (if necessary) must be started Index: pgpool-ja.html =================================================================== RCS file: /cvsroot/pgpool/pgpool-II/doc/pgpool-ja.html,v retrieving revision 1.129 diff -c -r1.129 pgpool-ja.html *** pgpool-ja.html 6 Jun 2011 08:18:55 -0000 1.129 --- pgpool-ja.html 19 Jun 2011 14:36:02 -0000 *************** *** 1967,1974 **** <h2>$B%/%(%j%-%c%C%7%e$N@_DjJ}K!(B</h2> <p> ! pgpool-II$B$G$O!"$9$Y$F$N%b!<%I$G%/%(%j%-%c%C%7%e$rMxMQ$9$k$3$H$,$G$-$^$9!#(B ! $BMxMQ$9$k>l9g$K$O!"(Bpgpool.conf$B$N@_Dj$r0J2<$N$h$&$K@_Dj$7$^$9!#(B <pre> enable_query_cache = true </pre> --- 1967,1981 ---- <h2>$B%/%(%j%-%c%C%7%e$N@_DjJ}K!(B</h2> <p> ! pgpool-II$B$G$O!"$9$Y$F$N%b!<%I$G%/%(%j%-%c%C%7%e$rMxMQ$9$k$3$H$,$G$-$^$9!#(B ! </p> ! ! <h3>SystemDB$B$r;HMQ$7$?%/%(%j%-%c%C%7%e(B</h3> ! ! <p> ! $BMxMQ$9$k>l9g$K$O!"(Bpgpool.conf$B$N@_Dj$r0J2<$N$h$&$K@_Dj$7$^$9!#(B ! </p> ! <pre> enable_query_cache = true </pre> *************** *** 1989,1994 **** --- 1996,2138 ---- $B$?$@$7!"$3$NNc$G$O%9%-!<%^L>$,(B"pgpool_catalog"$B$H$J$C$F$$$k$N$G!"0c$&%9%-!<%^$r;H$&>l9g$OE,Ev$K=q$-49$($F$/$@$5$$!#(B </p> + <h3>$B6&M-%a%b%j!&(BMemcached$B$r;HMQ$7$?%/%(%j%-%c%C%7%e(B</h3> + + <p> + $BMxMQ$9$k>l9g$K$O!"(Bpgpool.conf$B$N@_Dj$r0J2<$N$h$&$K@_Dj$7$^$9!#%G%U%)%k%H$O(Bfalse$B$G$9!#(B + </p> + + <pre> + memory_cache_enabled = true + </pre> + + <p> + $B6&M-%a%b%j!"$^$?$O(BMemcached$B$r;HMQ$7$?%/%(%j%-%c%C%7%e$N?6$kIq$$$O!"0J2<$N@_Dj$r9T$$$^$9!#(B + </p> + + <p> + $B%/%(%j%-%c%C%7%e$NJ}K!$O0J2<$N$h$&$K@_Dj$7$^$9!#(B<br> + $B6&M-%a%b%j$rMxMQ$9$k>l9g$O(B'shmem'$B!"(BMemcached$B$rMxMQ$9$k>l9g$O(B'memcached'$B$H@_Dj$7$^$9!#%G%U%)%k%H$O!"(B'shmem'$B$G$9!#(B + </p> + <pre> + memqcache_method = 'shmem' + </pre> + + <p> + Memcached$B$r;HMQ$9$k>l9g$O!"(BMemcached$B$N%[%9%HL>$r0J2<$N$h$&$K@_Dj$7$^$9!#(B + </p> + <pre> + memqcache_memcached_host = 'foo.com' + </pre> + + <p> + Memcached$B$r;HMQ$9$k>l9g$O!"(BMemcached$B$N%]!<%HHV9f$r0J2<$N$h$&$K@_Dj$7$^$9!#(B + </p> + <pre> + memqcache_memcached_port = 11211 + </pre> + + <p> + $B6&M-%a%b%j$r;HMQ$9$k>l9g$O!"%-%c%C%7%e$rC_$($k%a%b%j%5%$%:$r0J2<$N$h$&$K@_Dj$7$^$9!#C10L$O%P%$%H$G$9!#(B + </p> + <pre> + memqcache_total_size = 10240 + </pre> + + <p> + $B%/%(%j%-%c%C%7%e$N<wL?$O!"0J2<$N$h$&$K@_Dj$7$^$9!#C10L$OIC!"%G%U%)%k%H$O(B60$B$G$9!#(B + </p> + <pre> + memqcache_expire = 60 + </pre> + + <p> + $B%-%c%C%7%e$r5vMF$9$k(BSELECT$BJ8$N<B9T7k2L$N:GBg%5%$%:$O!"0J2<$N$h$&$K@_Dj$7$^$9!#C10L$O%P%$%H$G$9!#(B + </p> + <pre> + memqcache_maxcache = 512 + </pre> + + <p> + $B6&M-%a%b%j$r;HMQ$9$k>l9g$O!"%a%b%j%-%c%C%7%e$r8GDjD9$N%V%m%C%/$KJ,3d$7$^$9!#(B<br> + $B%V%m%C%/%5%$%:$r>.$5$/$9$k$H!"8zN(E*$J%a%b%j;HMQ$,=PMh$^$9$,!"=hM}B.EY$OCY$/$J$k62$l$,$"$j$^$9!#(B<br> + $B0lJ}!"%V%m%C%/%5%$%:$rBg$-$/$9$k$H!"8zN(E*$J%a%b%j;HMQ$O=PMh$^$;$s$,!"=hM}B.EY$N8~>e$,4|BT$G$-$^$9!#(B<br><br> + + $B%-%c%C%7%e%V%m%C%/%5%$%:$O0J2<$N$h$&$K@_Dj$7$^$9!#C10L$O%P%$%H$G!"%G%U%)%k%H$O(B8192$B$G$9!#(B + </p> + <pre> + memqcache_cache_block_size= 8192 + </pre> + + <h3>memcached$B$N%$%s%9%H!<%k(B</h3> + + <p> + $B%/%(%j%-%c%C%7%e$K$*$$$F!"(Bmemcached$B$r;HMQ$9$k>l9g$O!"(Bmemcached$B$r%$%s%9%H!<%k$7$F$*$/I,MW$,$"$j$^$9!#(B + </p> + + <p> + memcached$B$N%=!<%9%3!<%I$O(B<a href="http://memcached.org/";>memcached$B3+H/%Z!<%8(B</a>$B$+$i%@%&%s%m!<%I$G$-$^$9!#(B + </p> + + <dl> + <dt>configure$B$N<B9T(B + <dd> + <p> + $B%=!<%9%3!<%I$N(Btar ball$B$rE83+$7$?$i!"(Bconfigure$B$r<B9T$7$^$9!#(B + </p> + <pre> + ./configure + </pre> + </dd> + + <dt>make$B$N<B9T(B + <dd> + <pre> + make + make install + </pre> + </dd> + </dl> + + <h3>libmemcached$B$N%$%s%9%H!<%k(B</h3> + + <p> + memcached$B$N%/%i%$%"%s%H%i%$%V%i%j$O!"(Blibmemcached$B$r;HMQ$7$F$$$^$9!#(B<br> + memcached$B$N%$%s%9%H!<%k8e$K!"(Blibmemcached$B$r%$%s%9%H!<%k$9$kI,MW$,$"$j$^$9!#(B + </p> + + <p> + libmemcached$B$N%=!<%9%3!<%I$O!"(B<a href="http://libmemcached.org/libMemcached.html";>libMemcached$B3+H/%Z!<%8(B</a>$B$+$i%@%&%s%m!<%I$G$-$^$9!#(B + </p> + + <dl> + <dt>configure$B$N<B9T(B + <dd> + <p> + $B%=!<%9%3!<%I$N(Btar ball$B$rE83+$7$?$i!"(Bconfigure$B$r<B9T$7$^$9!#(B + </p> + <pre> + ./configure + </pre> + + <p> + configure$B$K;XDj$G$-$k%*%W%7%g%s$O0J2<$G$9!#(B + </p> + + <ul> + <li><code>--with-memcached=path</code><br/> + Memcached$B$,%$%s%9%H!<%k$5$l$F$$$k%H%C%W%G%#%l%/%H%j$r;XDj$7$^$9!#(B</li> + </ul> + </dd> + + <dt>make$B$N<B9T(B + <dd> + <pre> + make + make install + </pre> + </dd> + </dl> <h1>pgpool-II$B$N5/F0$HDd;_(B<a name="start"></a></h1> <p>
_______________________________________________ Pgpool-hackers mailing list [email protected] http://pgfoundry.org/mailman/listinfo/pgpool-hackers
