Hi!
On Feb 4, 2008, at 5:53 PM, Toru Maesaka wrote:
As for abstracting the storage behind a common interface, I'm
thinking that I should really forget about external engines for now
and concentrate on getting the current storage behind a interface,
that is to be used in a common fashion later on. Making this
loadable is probably not so difficult once we have this working. It
is definitely on my todo list :-)
Yep!
We defined the interface in MySQL pretty much based on the current
engines. We refactored from there.
This is counter to the belief that everything should be designed out
to perfection on the first pass... but frankly I hate this argument.
Do work, refactor work. I find it better to figure out the next X
steps and start the process of design then design everything from
start to finish (aka the Rational method).
int (*sb_settings)(void*, const char*);
where const char* can be something along the line of:
"database_name:option1=foo:option2=foo:option3=foo" (somewhat like
the DBI).
We will need a flexible way of passing options to engines. Either we
pipe unknown command line args, or we tell engines to define their own
configuration files.
Cheers,
-Brian
Not sure if this is a good solution but it looks promising in my head.
Any comments?
Toru
On Feb 5, 2008 2:51 AM, Trond Norbye <[EMAIL PROTECTED]> wrote:
Brian Aker wrote:
Hi!
I've been studying the patch from Toru Maesaka on making memcached
have a plugable storage backend:
http://alpha.mixi.co.jp/dist/memcached-1.2.4_modularexp-0.0.5.tar.gz
What I am wondering is how to get this into the main distribution.
I see some problems in the way that he went about what he did, but
mostly it looks good. What he has done is refactored the main
storage functions such that the function calls come from this
structure:
typedef struct {
void *opaque_engine; /* pointer to the database handle */
void *dlink_handler; /* dynamic library handler */
char *dboject_name; /* name of the db object to link to */
int max_dbsize; /* maximum database size */
int curr_size; /* current size of the database */
int bucket_num; /* optional value for buckets */
/* functions to be dynamically loaded */
void *(*ext_new)(void);
int (*ext_open)(void*, const char*);
void (*ext_close)(void*);
int (*ext_conf)(void*, const int, const char*);
void *(*ext_get)(void*, const void*, const int, int*);
int (*ext_put)(void*, const void*, const int, const void*, const
int);
int (*ext_del)(void*, const void*, const int);
int (*ext_flush)(void*, const char*);
} MMCSTORAGE;
He has left out increment/decrement, but I believe those would need
to be handled as well (if you want to push atomic operations into
the engine that is).
Thoughts? Toru's method involves if/else around calls, personally I
would prefer putting all logic behind the API.
I like this idea :-) I agree that we should not clutter the code
with if/else around the calls to the storage, so we should implement
a "default memory storage" that we use if no other is requested.
Just by looking at the struct, I feel that the first part of the
struct is specialized towards his needs and should be removed. I
would probably go for something like:
typedef struct {
int sb_version; /* to allow modifications to the structure */
/*
** The following member may be removed because the backend may
store it's internal data in it's own
** static struct. If we choose to remove it, we should aslo
remove the first parameter in the functions
** below
*/
void *sb_data; /* Pointer to a block where the storage backend
may store internal data */
/* The functions we need.. The first parameter should be sb_data
*/
void *(*sb_get)(void*, const void*, const int, int*);
int (*sb_put)(void*, const void*, const int, const void*, const
int);
int (*sb_del)(void*, const void*, const int);
int (*sb_flush)(void*, const char*);
...
} MMCSTORAGE;
Each "module" should contain the following functions:
MMCSTORAGE* create_instance(settings*, int version, int *error);
settings - The backend should be able to pick up the configuration
parameters it needs
version - The "protocol" version requested from the backend
error - A place the backend can store a predefined error code
void destroy_instance(MMCSTORAGE*);
memcached.c should include something like (not complete/compiled
code):
MMCSTORAGE* initialize_storage(const char *library, const char
*config)
{
MMCSTORAGE* ret;
void *handle = dlopen(library, RTLD_LAZY);
if (handle == NULL) {
/* ERROR, report it to the user */
return NULL;
}
MMCSTORAGE* (*create_instance)(void*, int, int*) = dlsym(handle,
"create_instance");
if (create_instance == NULL) {
/* ERROR, report it to the user */
return NULL;
}
int error;
/* request a instance with protocol version 1 */
ret = (*create_instance)(&settings, 1, &error);
if (ret == NULL) {
/* report this error message to the user */
}
return ret;
}
If no library is requested, we should use a built-in version. Well,
that's my ideas.. Comments anyone?
Trond
--
_______________________________________________________
Brian "Krow" Aker, brian at tangent.org
Seattle, Washington
http://krow.net/ <-- Me
http://tangent.org/ <-- Software
http://exploitseattle.com/ <-- Fun
_______________________________________________________
You can't grep a dead tree.
