(2010/04/13 4:28), Trond Norbye wrote:
>
> On 12. apr. 2010, at 02.18, KaiGai Kohei wrote:
>
>> Are you saying the data fragments are not necessary to be placed on the
>> continuous region in physically?
>> If so, it seems to me allocate() method needs to be revised to take an
>> arguments to indicate the number of fragments (with their lengths) on
>> allocation time?
>>
>> However, it seems to me the following description introduces all the
>> needed region is allocated on the allocation time at once, then the data
>> region will be segmented into several chunks.
>>
>
> Yes, the intention of the iovector is so that you can store the value for
> a key as multiple fragments that would be concatenated onto the wire in
> the return packet. This means that we could implement append / prepend
> really fast by just adding another element to the internal iovector of
> an item.
>
> We don't need to modify the allocate functions, because the core will call
> the method to get the iovector for the item when it needs to access the
> value.
>
> An important piece that is missing right now is the function the engine
> can call to get the maximum number of fragments that the core supports.
> I should get that in there as soon as possible.
Rather than the server API to inform maximum number of fragments,
it seems to me the get_item_info() should provides total number of
fragments in the specified item, and the memcached core calls engine
api to set up an iovector of the specified fragment of the item.
If so, a part of intermediating module will be able to drop a part of
the fragments which contains security context.
Of course, a large number of iteration of append/prepend may make
the data structure inefficient, so I think any mechanism to reorganize
well hashed fragments...
>> BTW, here is a point I missed it.
>> Currently, do_add_delta() and do_store_item() with APPEND/PREPEND option
>> assume the data field only contains the original data. So, if we inject
>> extra attributes into the data region, it performs unexpectedly.
>>
>> For example, if we append {key="aaa", value="mokemoke"} after the
>> {key="aaa", value="fugafuga"}, and both have "classified\0" as security
>> label, it will concatenate them without eliminating extra attributes,
>> like {key="aaa", value="fugafugaclassified\0mokemoke"}.
>> ^^^^^^^^^^^^
>> So, we need to inform secondary module where is the data segment and where
>> is the extra attributes of items, even if the secondary module allocate
>> data region in flat.
>
> If you're storing meta-data inside the real data of an item, you would just
> have to add your own implementation for append/prepend where you do the
> operation on the items and then try to call a cas set operation to the
> engine.
If we can store meta-data in a certain fragment, is it unnecessary to handle
append/preprend and delta operations in the intermediating module?
Thanks,
--
KaiGai Kohei <kai...@ak.jp.nec.com>
--
To unsubscribe, reply using "remove me" as the subject.
