Hi Kuer, Yes, this is the correct behavior. If there are still un-compacted key/value pairs in the CellCache, then the commit log fragments that they have been persisted in should not be removed.
- Doug On Tue, Jul 7, 2009 at 5:28 PM, kuer <[email protected]> wrote: > > Hi, Doug, > > Here is the source code of purge() : > > 227 int CommitLog::purge(int64_t revision) { > 228 ScopedLock lock(m_mutex); > 229 CommitLogFileInfo file_info; > 230 String fname; > 231 > 232 try { > 233 > 234 while (!m_fragment_queue.empty()) { > 235 file_info = m_fragment_queue.front(); > 236 fname = file_info.log_dir + file_info.num; > 237 if (file_info.revision <= revision) { > 238 m_fs->remove(fname); > 239 m_fragment_queue.pop_front(); > 240 HT_INFOF("clgc Removed log fragment file='%s' revision= > %lld", fname.c_str(), > 241 (Lld)file_info.revision); > 242 if (file_info.purge_log_dir) { > 243 HT_INFOF("Removing commit log directory %s", > file_info.log_dir.c_str()); > 244 m_fs->rmdir(file_info.log_dir); > 245 } > 246 } > 247 else { > 248 > 249 HT_WARNF("clgc LOG FRAGMENT PURGE breaking because %lld >= > %lld (file='%s')", > 250 (Lld)file_info.revision, (Lld)revision, > fname.c_str()); > 251 > 252 break; > 253 } > 254 } > 255 > 256 } > 257 catch (Hypertable::Exception &e) { > 258 HT_ERRORF("Problem purging log fragment fname = '%s'", > fname.c_str()); > 259 return e.code(); > 260 } > 261 > 262 return Error::OK; > 263 } > 264 > > When thread execute to LINE 249 & 250, it will break the while loop, > and return from purge(). In another word, It will not pop the fragment > in m_fragment_queue, and it still there. When purging next time, it > still handle the same fragment, and still break the while loop before > m_fragment_queue.empty(). > > In this situation, May I understand that there are always some > fragment in the m_fragment_queue which never be purged ??? > > -- kuer > > On 7月8日, 上午7时54分, Doug Judd <[email protected]> wrote: > > Hi kuer, > > > > I don't see any problem with the log snippet that you give below. The > "clgc > > LOG FRAGMENT PURGE" messages are just leftover debugging statements that > I > > forgot to remove in a recent checkin. Are you experiencing a real > problem > > with the database? If so, what is the symptom? > > > > - Doug > > > > On Tue, Jul 7, 2009 at 4:48 PM, kuer <[email protected]> wrote: > > > > > I found lots of message in rangeserver's log : > > > > > 2009-07-08 07:30:23,198 1231112512 Hypertable.RangeServer [WARN] (Lib/ > > > CommitLog.cc:250) clgc LOG FRAGMENT PURGE breaking because > > > 1247008867417054087 >= 1247003041037972148 (file='/hypertable/servers/ > > > 221.194.134.176_31060/log/user/75') > > > 2009-07-08 07:30:23,198 1231112512 Hypertable.RangeServer [INFO] > > > (RangeServer/RangeServer.cc:2032) Memory Usage: 322964958 bytes > > > 2009-07-08 07:30:43,197 1336011072 Hypertable.RangeServer [INFO] > > > (RangeServer/RangeServer.cc:2032) Memory Usage: 323476974 bytes > > > 2009-07-08 07:31:03,198 1199642944 Hypertable.RangeServer [WARN] (Lib/ > > > CommitLog.cc:250) clgc LOG FRAGMENT PURGE breaking because > > > 1247008867417054087 >= 1247003041037972148 (file='/hypertable/servers/ > > > 221.194.134.176_31060/log/user/75') > > > 2009-07-08 07:31:03,198 1199642944 Hypertable.RangeServer [INFO] > > > (RangeServer/RangeServer.cc:2032) Memory Usage: 323476974 bytes > > > 2009-07-08 07:31:23,197 1294051648 Hypertable.RangeServer [INFO] > > > (RangeServer/RangeServer.cc:2032) Memory Usage: 323476974 bytes > > > 2009-07-08 07:31:43,198 1178663232 Hypertable.RangeServer [WARN] (Lib/ > > > CommitLog.cc:250) clgc LOG FRAGMENT PURGE breaking because > > > 1247008867417054087 >= 1247003041037972148 (file='/hypertable/servers/ > > > 221.194.134.176_31060/log/user/75') > > > 2009-07-08 07:31:43,198 1178663232 Hypertable.RangeServer [INFO] > > > (RangeServer/RangeServer.cc:2032) Memory Usage: 324501006 bytes > > > > > It seems that it will be never end. > > > > > I want to know : > > > 1. How this happened? why to purge a future revision? > > > 2. How to resolve/correct this problem? > > > From the purge() function, the thread will stop purging if > > > meeting with the 'fucture' revision, and purge() will end. The > > > 'future' revision data is still there; when purging next time, it will > > > complain it again. > > > What's the rationale behind this? > > > Would someone tell me kindly? > > > > > Thanks. > > > > > -- kuer > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en -~----------~----~----~----~------~----~------~--~---
