Hi Kuer,
I suspect the BloomFilter code is fine. Taking a look at your logs it
looks like the METADATA table is going through a split and the
AccessGroup "logging" is undergoing a major compaction, however since
it is empty, nothing gets inserted in BloomFilterItems and hence the
assert gets hit.
(From the log:
2009-07-22 09:39:01,415 1351514432 Hypertable.RangeServer [INFO]
(RangeServer/AccessGroup.cc:379) Starting Major Compaction of
METADATA[0:<FF> <FF>..<FF><FF>](logging))
This AccessGroup is currently not used by the system and so it should
be empty and should not undergo a compaction. Can you make the change
(below) to AccessGroup.cc so we can have a better idea of why its
compacting? I suspect it might be a memory corruption issue (that we
have a fix for in the upcoming release).
It would be great if you ran the RangeServer with valgrind turned on
and see if the valgrind log reveals anything further (to do this add
the option --valgrind-rangeserver when calling the start-all-
servers.sh script (eg: <$HYPERTABLE_INSTALL_DIR/bin/start-all-
servers.sh --valgrind-rangeserver > ))
-Sanjit
--- a/src/cc/Hypertable/RangeServer/AccessGroup.cc
+++ b/src/cc/Hypertable/RangeServer/AccessGroup.cc
@@ -375,8 +375,8 @@ void AccessGroup::run_compaction(bool major) {
if (m_immutable_cache->memory_used()==0 && m_stores.size()
<= (size_t)1)
HT_THROW(Error::OK, "");
tableidx = 0;
- HT_INFOF("Starting Major Compaction of %s(%s)",
- m_range_name.c_str(), m_name.c_str());
+ HT_INFOF("Starting Major Compaction of %s(%s) immutable cache
mem=%llu, num cell stores=%d",
+ m_range_name.c_str(), m_name.c_str(),
m_immutable_cache->memory_used(), m_stores.size());
}
else {
if (m_stores.size() >
(size_t)Global::access_group_max_files) {
On Jul 22, 2009, at 4:11 AM, kuer wrote:
>
> Hi, all,
>
> I find something interesting in cc/Hypertable/RangeServer/
> CellStoreV1.cc :
>
> 168 if (m_bloom_filter_mode != BLOOM_FILTER_DISABLED) {
> 169 m_bloom_filter_items = new BloomFilterItems(); // aproximator
> items
> 170 }
>
> 367
> 368 // if bloom_items haven't been spilled to create a bloom filter
> yet, do it
> 369 if (m_bloom_filter_mode != BLOOM_FILTER_DISABLED) {
> 370 if (m_bloom_filter_items) {
> ^^^^^^^^^^^^^^^^^^^^^^^^^^
> I think this cannot promise m_bloom_filter_items->size() > 0
>
> 371 m_trailer.num_filter_items = m_bloom_filter_items->size();
> ^^^^^ How about adding the following lines ???
> +
> + if (m_trailer.num_filter_items < 1 ) {
> + m_trailer.num_filter_items = m_max_entries;
> + }
> + if (m_trailer.num_filter_items < 1) {
> + m_trailer.num_filter_items = 1;
> + }
> +
> 372 create_bloom_filter();
> 373 }
> 374 assert(!m_bloom_filter_items && m_bloom_filter);
> 375
> 376 m_bloom_filter->serialize(send_buf);
> 377 m_filesys->append(m_fd, send_buf, 0, &m_sync_handler);
> 378
> 379 m_outstanding_appends++;
> 380 m_offset += m_bloom_filter->size();
> 381 }
> 382
>
>
> thanks
>
> -- kuer
>
>
> On 7月22日, 下午5时05分, kuer <[email protected]> wrote:
>> Hi, all,
>>
>> the content of the file that cause assertion failure of BloomFilter :
>>
>> /hypertable/tables/METADATA/logging/AB2A0D28DE6B77FFDD6C72AF/cs0
>>
>> $ hexdump -C cs0
>> 00000000 49 64 78 46 69 78 2d 2d 2d 2d 1a 00 ff ff ff ff |
>> IdxFix----......|
>> 00000010 00 00 00 00 00 00 00 00 7d 9f 49 64 78 56 61 72
>> |........}.IdxVar|
>> 00000020 2d 2d 2d 2d 1a 00 ff ff ff ff 00 00 00 00 00 00
>> |----............|
>> 00000030 00 00 87 97 |....|
>> 00000034
>>
>> FYI
>>
>> -- kuer
>>
>> On 7月22日, 下午1时03分, Sanjit Jhala <[email protected]>
>> wrote:
>>
>>> Recovering ranges from crashed RangeServers is one of the high
>>> priority items Doug is working on.
>>
>>> -Sanjit
>>
>
> >
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Hypertable Development" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/hypertable-dev?hl=en
-~----------~----~----~----~------~----~------~--~---
