Hello I wrote documents about query cache (shmem or memcached). I'm sending the following patch file. Your comment would be appreciated.
- pgpool-ja.html - pgpool-en.html Regards Masanori YAMAZAKI 2011/6/16 Masanori Yamazaki <[email protected]> > I fixed errors and these program can be compiled. > > Some functions is declared in header file. Take memqcache.h for example. > Other c files include memqcache.h in order to use functions in memqcache.c. > (Refer to pool_process_context.c, pool_process_context.h) > > [memqcache.h] > extern POOL_STATUS memqcache_lookup(POOL_ > CONNECTION *frontend, POOL_CONNECTION_POOL *backend, Node *node); > extern int set_cache_on_memcached(POOL_CONNECTION *frontend, const char > *md5_query, const char *data); > extern int get_cache_on_memcached(POOL_CONNECTION *frontend, const char > *md5_query, char **buf, size_t *len); > extern int delete_cache_on_memcached(const char *key); > ...and so on. > > Pgpool.conf.sample is included. > > In pool_memqcache.c, I removed header file which is not being used. > > Static functions is declared before the function body is defined. > > > > memcached.h and related stuffs should not be included > > unconditionaly. I prefer to enable memcached in > > configure(i.e. configure --with-memcached=... style). Pgpool > > hackers, I would like to hear opinions for this though. > > Now memcached.h is included in pool.h. > > "#include <libmemcached/memcached.h>" > > I implemented it without minding problem. > If need to take a method of doing "configure", I do so. > > Now I am writing documents(doc/pgpool-en.html and doc/pgpool-ja.html). > It will take more time to complete it. > > > Thanks > Masanori YAMAZAKI > > > > > 2011/6/15 Masanori Yamazaki <[email protected]> > >> 2011/6/15 Tatsuo Ishii <[email protected]> >> >>> Some comments to this patch: >>> >>> - Static functions should be declared before the function body is >>> defined. >>> >> I correct it. >> >> >>> - What memcache library are you using? There are some and you do not >>> explain which one you are using. Maybe this? >>> >>> http://tangent.org/552/libmemcached.html >>> >>> >> Yes, memcached library I am using is "libmemcached".Sorry I forgot to >> explain it. >> http://libmemcached.org/libMemcached.html >> >> >> >>> - memcached.h and related stuffs should not be included >>> unconditionaly. I prefer to enable memcached in >>> configure(i.e. configure --with-memcached=... style). Pgpool >>> hackers, I would like to hear opinions for this though. >>> >>> >> - No pgpool.conf.sample* included >>> >> I correct it >> >> >>> - No documents(doc/pgpool-en.html and doc/pgpool-ja.html) >>> included. These should be provided in the first place. >>> >> >> I add what is implemented to these documents. >> >> >> >>> - Did not compile: >>> >>> gcc -DHAVE_CONFIG_H -DDEFAULT_CONFIGDIR=\"/usr/local/etc\" -I. >>> -D_GNU_SOURCE -I /usr/local/pgsql/include -g -O2 -Wall >>> -Wmissing-prototypes -Wmissing-declarations -MT child.o -MD -MP -MF >>> .deps/child.Tpo -c -o child.o child.c >>> child.c: In function 'do_child': >>> child.c:153: warning: implicit declaration of function >>> 'init_memcached_connection' >>> child.c: At top level: >>> child.c:1966: warning: conflicting types for 'init_memcached_connection' >>> child.c:1966: error: static declaration of 'init_memcached_connection' >>> follows non-static declaration >>> child.c:153: error: previous implicit declaration of >>> 'init_memcached_connection' was here >>> child.c: In function 'init_memcached_connection': >>> child.c:1969: warning: implicit declaration of function >>> 'memcached_connect' >>> child.c:1970: error: 'memcached_con' undeclared (first use in this >>> function) >>> child.c:1970: error: (Each undeclared identifier is reported only once >>> child.c:1970: error: for each function it appears in.) >>> make[2]: *** [child.o] Error 1 >>> make[2]: Leaving directory `/home/t-ishii/work/pgfoundry/GSoC/pgpool-II' >>> make[1]: *** [all-recursive] Error 1 >>> make[1]: Leaving directory `/home/t-ishii/work/pgfoundry/GSoC/pgpool-II' >>> make: *** [all] Error 2 >>> >> >> I correct an error of program and send new patch file as soon as possible. >> >> comment thanks >> >> >>> -- >>> Tatsuo Ishii >>> SRA OSS, Inc. Japan >>> English: http://www.sraoss.co.jp/index_en.php >>> Japanese: http://www.sraoss.co.jp >>> >>> > 2011/6/14 Tatsuo Ishii <[email protected]> >>> > >>> >> Please explain what is implemented and what is not implemented in this >>> >> patch. >>> >> >>> > >>> > What is implemented in this patch are as below. >>> > - connect to memcached. >>> > - set RowDescription or DataRow on memcached. >>> > - get RowDescription or DataRow on memcached by using key of md5(SELECT >>> > convert). >>> > - delete cache on memcached by using key of md5(SELECT convert). only >>> > module. >>> > - fetch and parse the fellowing data in pgpool.conf. >>> > >>> > memory_cache_enabled >>> > memqcache_method >>> > memqcache_memcached_host >>> > memqcache_memcached_port >>> > memqcache_total_size >>> > memqcache_expire >>> > memqcache_maxcache >>> > memqcache_cache_block_size >>> > >>> > - add data to buffer before set data on memcached. >>> > - get buffer in order to set it on memcached >>> > - initialize buffer >>> > >>> > I am implemented only in Simple Query case. >>> > >>> > >>> > What is not implemented in this patch are as below. >>> > - shmem cache. >>> > structure of managing table oids. >>> > the space management map. >>> > on memory query cache structure. >>> > >>> > - implementation in Extended Query case. >>> > >>> > - cache validation. >>> > The result of SELECT using none IMMUTABLE functions is not cached. >>> > >>> > - remove cache >>> > If cache is full, the least recently used cache is removed. >>> > If a table is dropped or modified, related cache data is deleted. >>> > If a table definition is modified(ALTER TABLE), related cache data is >>> > deleted. >>> > If a schema or database is dropped, related cache data is deleted. >>> > >>> > >>> > This week I am mainly implementing the shmem cache. >>> > >>> > Regards >>> > Masanori YAMAZAKI >>> > >>> > >>> > >>> > >>> >> -- >>> >> Tatsuo Ishii >>> >> SRA OSS, Inc. Japan >>> >> English: http://www.sraoss.co.jp/index_en.php >>> >> Japanese: http://www.sraoss.co.jp >>> >> >>> >> > Hello, hackers >>> >> > >>> >> > I have implemented a function caching query on memcached. >>> >> > I am sending the patch file and it's so nice to give me some >>> opinions. >>> >> > >>> >> > Main program which I implemented is as below. >>> >> > - Search for cache data on memcached by using md5 which converted >>> SELECT >>> >> > into. >>> >> > - Set data on memcached. The key is md5 which converted SELECT >>> into. >>> >> > The value is RowDescription, DataRow from Backend. >>> >> > >>> >> > [Modify files] >>> >> > - pool.h >>> >> > - child.c >>> >> > - pool_config.l >>> >> > - pool_config.h >>> >> > - pool_config.c >>> >> > - pool_system.c >>> >> > - pool_proto_modules.c >>> >> > - pool_process_query.c >>> >> > >>> >> > [New File] >>> >> > - pool_memqcache.c >>> >> > >>> >> > >>> >> > -- >>> >> > Regards >>> >> > Masanori YAMAZAKI >>> >> >>> >> >> >
Index: pgpool-ja.html =================================================================== RCS file: /cvsroot/pgpool/pgpool-II/doc/pgpool-ja.html,v retrieving revision 1.129 diff -c -r1.129 pgpool-ja.html *** pgpool-ja.html 6 Jun 2011 08:18:55 -0000 1.129 --- pgpool-ja.html 16 Jun 2011 14:50:28 -0000 *************** *** 1967,1974 **** <h2>$B%/%(%j%-%c%C%7%e$N@_DjJ}K!(B</h2> <p> ! pgpool-II$B$G$O!"$9$Y$F$N%b!<%I$G%/%(%j%-%c%C%7%e$rMxMQ$9$k$3$H$,$G$-$^$9!#(B ! $BMxMQ$9$k>l9g$K$O!"(Bpgpool.conf$B$N@_Dj$r0J2<$N$h$&$K@_Dj$7$^$9!#(B <pre> enable_query_cache = true </pre> --- 1967,1981 ---- <h2>$B%/%(%j%-%c%C%7%e$N@_DjJ}K!(B</h2> <p> ! pgpool-II$B$G$O!"$9$Y$F$N%b!<%I$G%/%(%j%-%c%C%7%e$rMxMQ$9$k$3$H$,$G$-$^$9!#(B ! </p> ! ! <h3>SystemDB$B$r;HMQ$7$?%/%(%j%-%c%C%7%e(B</h3> ! ! <p> ! $BMxMQ$9$k>l9g$K$O!"(Bpgpool.conf$B$N@_Dj$r0J2<$N$h$&$K@_Dj$7$^$9!#(B ! </p> ! <pre> enable_query_cache = true </pre> *************** *** 1989,1994 **** --- 1996,2068 ---- $B$?$@$7!"$3$NNc$G$O%9%-!<%^L>$,(B"pgpool_catalog"$B$H$J$C$F$$$k$N$G!"0c$&%9%-!<%^$r;H$&>l9g$OE,Ev$K=q$-49$($F$/$@$5$$!#(B </p> + <h3>$B6&M-%a%b%j!&(BMemcached$B$r;HMQ$7$?%/%(%j%-%c%C%7%e(B</h3> + + <p> + $BMxMQ$9$k>l9g$K$O!"(Bpgpool.conf$B$N@_Dj$r0J2<$N$h$&$K@_Dj$7$^$9!#%G%U%)%k%H$O(Bfalse$B$G$9!#(B + </p> + + <pre> + memory_cache_enabled = true + </pre> + + <p> + $B6&M-%a%b%j!"$^$?$O(BMemcached$B$r;HMQ$7$?%/%(%j%-%c%C%7%e$N?6$kIq$$$O!"0J2<$N@_Dj$r9T$$$^$9!#(B + </p> + + <p> + $B%/%(%j%-%c%C%7%e$NJ}K!$O0J2<$N$h$&$K@_Dj$7$^$9!#(B<br> + $B6&M-%a%b%j$rMxMQ$9$k>l9g$O(B'shmem'$B!"(BMemcached$B$rMxMQ$9$k>l9g$O(B'memcached'$B$H@_Dj$7$^$9!#%G%U%)%k%H$O!"(B'shmem'$B$G$9!#(B + </p> + <pre> + memqcache_method = 'shmem' + </pre> + + <p> + Memcached$B$r;HMQ$9$k>l9g$O!"(BMemcached$B$N%[%9%HL>$r0J2<$N$h$&$K@_Dj$7$^$9!#(B + </p> + <pre> + memqcache_memcached_host = 'foo.com' + </pre> + + <p> + Memcached$B$r;HMQ$9$k>l9g$O!"(BMemcached$B$N%]!<%HHV9f$r0J2<$N$h$&$K@_Dj$7$^$9!#(B + </p> + <pre> + memqcache_memcached_port = 11211 + </pre> + + <p> + $B6&M-%a%b%j$r;HMQ$9$k>l9g$O!"%-%c%C%7%e$rC_$($k%a%b%j%5%$%:$r0J2<$N$h$&$K@_Dj$7$^$9!#C10L$O%P%$%H$G$9!#(B + </p> + <pre> + memqcache_total_size = 10240 + </pre> + + <p> + $B%/%(%j%-%c%C%7%e$N<wL?$O!"0J2<$N$h$&$K@_Dj$7$^$9!#C10L$OIC!"%G%U%)%k%H$O(B60$B$G$9!#(B + </p> + <pre> + memqcache_expire = 60 + </pre> + + <p> + $B%-%c%C%7%e$r5vMF$9$k(BSELECT$BJ8$N<B9T7k2L$N:GBg%5%$%:$O!"0J2<$N$h$&$K@_Dj$7$^$9!#C10L$O%P%$%H$G$9!#(B + </p> + <pre> + memqcache_maxcache = 512 + </pre> + + <p> + $B6&M-%a%b%j$r;HMQ$9$k>l9g$O!"%a%b%j%-%c%C%7%e$r8GDjD9$N%V%m%C%/$KJ,3d$7$^$9!#(B<br> + $B%V%m%C%/%5%$%:$r>.$5$/$9$k$H!"8zN(E*$J%a%b%j;HMQ$,=PMh$^$9$,!"=hM}B.EY$OCY$/$J$k62$l$,$"$j$^$9!#(B<br> + $B0lJ}!"%V%m%C%/%5%$%:$rBg$-$/$9$k$H!"8zN(E*$J%a%b%j;HMQ$O=PMh$^$;$s$,!"=hM}B.EY$N8~>e$,4|BT$G$-$^$9!#(B<br><br> + + $B%-%c%C%7%e%V%m%C%/%5%$%:$O0J2<$N$h$&$K@_Dj$7$^$9!#C10L$O%P%$%H$G!"%G%U%)%k%H$O(B8192$B$G$9!#(B + </p> + <pre> + memqcache_cache_block_size= 8192 + </pre> <h1>pgpool-II$B$N5/F0$HDd;_(B<a name="start"></a></h1> <p> Index: pgpool-en.html =================================================================== RCS file: /cvsroot/pgpool/pgpool-II/doc/pgpool-en.html,v retrieving revision 1.91 diff -c -r1.91 pgpool-en.html *** pgpool-en.html 1 May 2011 09:41:48 -0000 1.91 --- pgpool-en.html 16 Jun 2011 14:50:30 -0000 *************** *** 2050,2057 **** </p> <h2>Setting Query cache method</h2> ! <p>The Query cache can be used in all modes in pgpool-II. Activating it in ! pgpool.conf is done as follows:</p> <pre> enable_query_cache = true </pre> --- 2050,2063 ---- </p> <h2>Setting Query cache method</h2> ! <p>The Query cache can be used in all modes in pgpool-II.</p> ! ! <h3>Using SystemDB</h3> ! ! <p> ! Activating it in pgpool.conf is done as follows: ! </p> ! <pre> enable_query_cache = true </pre> *************** *** 2074,2079 **** --- 2080,2149 ---- "pgpool_catalog". </p> + <h3>Using Shared memory or Memcached</h3> + + <p> + To enable the memory cache functionality, set this to true (default is false) + </p> + + <pre> + memory_cache_enabled = true + </pre> + + <p> + Memory cache behavior can be specified by cache_param directive:<br> + Cache store method. Either "shmem"(shared memory) or "memcached". Default is shmem. + </p> + <pre> + memqcache_method = 'shmem' + </pre> + + <p> + Memcached host name. Mandatory if memqcache_method=memcached. If this parameter is starting with "/", UNIX domain socket is assumed. + </p> + <pre> + memqcache_memcached_host = 'foo.com' + </pre> + + <p> + Memcached port number. Mondatory if memqcache_method=memcached. + </p> + <pre> + memqcache_memcached_port = 11211 + </pre> + + <p> + Total memory size in bytes for storing memory cache. Mandatory if memqcache_method=shmem. + </p> + <pre> + memqcache_total_size = 10240 + </pre> + + <p> + Memory cache entry life time specified in seconds. Default is 60. + </p> + <pre> + memqcache_expire = 60 + </pre> + + <p> + Maximum SELECT result size in bytes. If exceeds this value, the result is not cached. + </p> + <pre> + memqcache_maxcache = 512 + </pre> + + <p> + Cache block size in bytes<br> + Only meaning full when memqcache_method=shmem. Memory cache is devided into fixed size of block.<br> + If the block size is small, memory usage is efficient, but processing might be slow. On the other hand,<br> + if the block size is large, memory usage is not efficient but processing might be fast. Default is 8192. <br> + + </p> + <pre> + memqcache_cache_block_size= 8192 + </pre> + <h1>Starting/Stopping pgpool-II<a name="start"></a></h1> <p>All the backends and the System DB (if necessary) must be started
_______________________________________________ Pgpool-hackers mailing list [email protected] http://pgfoundry.org/mailman/listinfo/pgpool-hackers
