Which hypertable version? (from bin/hypertable --version). The latest version fixed a couple corruption issues.
On Sat, Aug 1, 2009 at 8:01 AM, kuer<[email protected]> wrote: > > Hi, all, > > In my testing environment, there are 4 kfs-chunkserver + hypertable- > rangeserver. > > The rangeserver on the 2nd complained something, and the client got > stuck with feeding data into hypertable. > > Here are the error logs of rangeserver in 2nd box : > > 2009-08-01 21:52:05,363 1352718656 Hypertable.RangeServer [ERROR] > operator() (RangeServer/MaintenanceQueue.h:126): > Hypertable::Exception: > shrink failed - DFS BROKER bad filename > at void Hypertable::AccessGroup::shrink(Hypertable::String&, > bool) > (RangeServer/AccessGroup.cc:634) > at virtual int64_t Hypertable::DfsBroker::Client::length(const > Hypertable::String&) (Lib/Client.cc:431): Error getting length of > DFS file: > /hypertable/tables/storage_se/agPORT/B6B3D15F8A7C517A21353A32/cs11 > at virtual int64_t Hypertable::DfsBroker::Client::length(const > Hypertable::String&) (Lib/Client.cc:425): No such file or > directory > 2009-08-01 21:52:05,363 1352718656 Hypertable.RangeServer [ERROR] > (RangeServer/MaintenanceQueue.h:138) Maintenance Task 'SPLIT > storage_se[cc.changjiang.www/bbs/viewthread.php?tid=313155&extra=page > %3D1..cc.dazhihui.bbs/thread-130962-1-1.html]' > failed, dropping task ... > > > I didnot find any other error meaningful message in loggings of other > hypertable or KFS process. > > Firstly, I restarted the ThriftBroker process, but it did NOT change > the situation. So I restarted Hypertable.RangeServer, bad news > came : > > 2009-08-01 22:09:12,870 47660526791088 Hypertable.RangeServer [INFO] > (RangeServer/Range.cc:231) Loading CellStore > /hypertable/tables/storage_se/agPORT/AB2A0D28DE6B77FFDD6C72AF/cs65 > 2009-08-01 22:09:12,875 47660526791088 Hypertable.RangeServer [ERROR] > replay_load_range (RangeServer/RangeServer.cc:1753): > Hypertable::Exception: Problem reading trailer for CellStore file > '/hypertable/tables/storage_se/agPORT/AB2A0D28DE6B77FFDD6C72AF/cs65' - > only read 0 of 512 bytes - DFS BROKER i/o error > at static Hypertable::CellStore* > Hypertable::CellStoreFactory::open(const Hypertable::String&, > const char*, const char*) (RangeServer/CellStoreFactory.cc:63) > 2009-08-01 22:09:12,875 47660526791088 Hypertable.RangeServer [INFO] > (RangeServer/RangeServer.cc:408) replay log > log_dir=/hypertable/servers/221.194.134.174_31060/log/user > 2009-08-01 22:09:14,495 47660526791088 Hypertable.RangeServer [FATAL] > (Lib/CommitLogBlockStream.cc:116) failed expectation: nread == > header->get_data_zlength() > > Now, the rangeserver cored and would not come back. > > I guess : > 1. maybe restarting operation lose some data in KFS > 2. maybe some error destroyed data in KFS before restarting > > I wish the system can work, even losing some data that cause the > disaster. > How to do? > > thanks > > -- kuer > > > > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en -~----------~----~----~----~------~----~------~--~---
