Djellel Eddine Difallah (GSOC): dedcode
It has been a while since I didn’t wrote on my GS0C project “Query Cache”, though many advancements have been done. So far I can say that I have a much clearer idea of the query processing flow and the different data structures used. I have been able to write a first plugin that can actually cache a query into memcached and keep some metadata on a local map object. Another cool option is to use data dictionary views to check int real time the cache content (thanks to the very easy interface). The next step from here is to write the retrieval functionality, this is probably the most difficult part for me cuz I still need to understand the functioning of the client interface.
Before making any further step, here is wiki I have posted on the of drizzle’s wiki page. It is basically an explanation of the query cache plugin interface and the hooks, this has yet to be approved and merged into drizzle trunk.
== Principle of the Query Cache ==
The Query Cache permits to return the results of a query from a cache repository if the query has been already cached; or, to add its results to a cache repository if it’s not cached yet. Therefore the simplest capability that a Query Cache plugin interface needs to provide is the ability to say if the received query is cached or not.
Then:
If the query is cached:
- Retrieve the resultset from the cache and skip the query parsing/processing
If the query is not cached:
- Initiate the Query Cache plugin, where the specific plugin will:
- Check if the query is cacheable
- Reserve a temp area and return a pointer to the session
- Build the hash key
- etc
- Add the results to the current temp area.
- Push the the temp variable and the respective cache key to the cache.
== The Query Cache plugin Interface design ==
In the current design we have introduced the following functions to the interface (drizzled/plugin/query_cache.h)
=== static bool isCached(Session *session); ===
This method is responsible to probe the used cache system for an entry corresponding to the received query.
”Note: The plugin is responsible to check if the query is a Select.”
=== Static bool sendCachedResultset(Session *session); ===
This method is called directly if the query is cached and skip the lexical and query processing.
=== static bool prepareResultset(Session *session); ===
This method is called before the query handling (see: handle_select() method) in order to prepare the resultset that will receive the generated data. An opened session will have its own resultset variable and current query key, the plugin will have to initiate these two variables added to the Class Session.
=== static bool insertRecord(Session *session, List<Item> &item); ===
The result rows generated will ultimately go though the client interface, the query cache system intercept these calls and populate the resultset with the data being send to the client (see: select_send() method)
=== static bool setResultset(Session *session); ===
Once the query processing is terminated this method will send the resultset variable to the cache repository and finalize the environment variables.
The query resultset being stored and populated in the session is of type Google Protobuf:
drizzled/message/resultset.proto
= Plugin Hooks =
The following branch contains the plugin interface hooks in the kernel drizzle:
[lp:~dedzone/drizzle/query-cache-hook/changes]
== The query interception ==
Upon reception of a query, (sql_parse.cc:717) in the mysql_parse method, and before even proceeding to the lexical and query processing, a call to ”isCached(session)” is made, a session pointer is passed for the query text, schema, and may be other parameters (for future use).
If isCached returns ”’true”’ then we’ll try to retrieve the results from the cache with ”sendCachedResultset”. If the retrieving operation succeeds (returns false) then we skip the remaining steps.
In the contrary – ie:
- If fetching the results fails or
- If the query is not cached
Then => proceed normally with the query parsing and processing
<pre>
void mysql_parse(Session *session, const char *inBuf, uint32_t length)
{
lex_start(session);
session->reset_for_next_command();
/* Check if the Query is Cached if and return true if yes
* TODO the plugin has to make sure that the query is cacheble
* by setting the query_safe_cache param to TRUE
*/
bool res= true;
if (plugin::QueryCache::isCached(session))
{
res= plugin::QueryCache::sendCachedResultset(session);
}
if (not res)
{
#if defined(DEBUG)
errmsg_printf(ERRMSG_LVL_DBUG,_(“Results retrieved from cache”));
#endif /* DEBUG */
return;
}
LEX *lex= session->lex;
Lex_input_stream lip(session, inBuf, length);
bool err= parse_sql(session, &lip);
……..
……..
lex->unit.cleanup();
session->set_proc_info(“freeing items”);
session->end_statement();
session->cleanup_after_query();
}
</pre>
== The Cache preparation and reception of the Resulset ==
In our journey executing a query, if the query is a Select will land in the (sql_parse.cc:494) execute_sqlcom_select method, this is basically the entry point of the query processing because it prepares to launch the handle_select method. before making this call will insert our prepareResultset hook, which will instruct the plugin implementation to:
- Prepare a cache to receive the resultset, and link it to the session::resultset pointer.
- create a hash key for our query, and link it to the session::query_cache_key member.
At this point the Session will be ‘aware’ that there is a plugin trying to cache the results of the query.
Immediatly after executing the handle_select, where all the resultset were generated, send to the client and added to the current resultset (As we shall see in the next session), we make a setResultset call, this will:
- Push the resultset to the cache area initiated by the plugin
- Reset the session members, resultset and query_cache_key.
<pre>
bool execute_sqlcom_select(Session *session, TableList *all_tables)
{
LEX *lex= session->lex;
select_result *result=lex->result;
bool res= false;
…..
…..
if (!result && !(result= new select_send()))
return true;
/* Init the Query Cache plugin */
plugin::QueryCache::prepareResultset(session);
res= handle_select(session, lex, result, 0);
/* Send the Resultset to the cache */
plugin::QueryCache::setResultset(session);
if (result != lex->result)
delete result;
}
}
return res;
}
</pre>
== Populating the resultset ==
The idea behind adding data to the resultset is to intercept the send_data() method in (select_send.h:87) When sending a row to the client interface. So the hook will just pass the row (items) to the query cache plugin using insertResultset(session, items).
<pre>
/* Send data to client. Returns 0 if ok */
bool send_data(List<Item> &items)
{
…….
…….
while ((item=li++))
{
if (item->send(session->client, &buffer))
{
my_message(ER_OUT_OF_RESOURCES, ER(ER_OUT_OF_RESOURCES), MYF(0));
break;
}
}
/* Insert this record to the Resultset into the cache */
if (session->query_cache_key != “” && session->getResultsetMessage() != NULL)
plugin::QueryCache::insertRecord(session, items);
session->sent_row_count++;
……
}
</pre>
URL: http://dedcode.wordpress.com/2010/07/11/20/
_______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
