Author: kpvdr Date: Tue Apr 1 20:33:15 2014 New Revision: 1583778 URL: http://svn.apache.org/r1583778 Log: QPID-5362: Bugfix for qpid_qls_analyze
Modified: qpid/trunk/qpid/cpp/src/linearstore.cmake qpid/trunk/qpid/cpp/src/qpid/linearstore/ISSUES qpid/trunk/qpid/tools/src/py/qls/jrnl.py Modified: qpid/trunk/qpid/cpp/src/linearstore.cmake URL: http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/src/linearstore.cmake?rev=1583778&r1=1583777&r2=1583778&view=diff ============================================================================== --- qpid/trunk/qpid/cpp/src/linearstore.cmake (original) +++ qpid/trunk/qpid/cpp/src/linearstore.cmake Tue Apr 1 20:33:15 2014 @@ -36,6 +36,9 @@ else (DEFINED linearstore_force) CHECK_LIBRARY_EXISTS (aio io_queue_init "" HAVE_AIO) CHECK_INCLUDE_FILES (libaio.h HAVE_AIO_H) if (HAVE_AIO AND HAVE_AIO_H) + # + # allow linearstore to be built + # message(STATUS "BerkeleyDB for C++ and libaio found, Linearstore support enabled") set (linearstore_default ON) else (HAVE_AIO AND HAVE_AIO_H) Modified: qpid/trunk/qpid/cpp/src/qpid/linearstore/ISSUES URL: http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/src/qpid/linearstore/ISSUES?rev=1583778&r1=1583777&r2=1583778&view=diff ============================================================================== --- qpid/trunk/qpid/cpp/src/qpid/linearstore/ISSUES (original) +++ qpid/trunk/qpid/cpp/src/qpid/linearstore/ISSUES Tue Apr 1 20:33:15 2014 @@ -39,21 +39,14 @@ Current/pending: svn r.1560530 2014-01-22: Bugfixes for qpid_qls_analyze svn r.1561848 2014-01-27: Bugfixes and enhancements for qpid_qls_analyze svn r.1564808 2014-02-05: Bugfixes and enhancements for qpid_qls_analyze + svn r.1578899 2014-03-18: Bugfixes and enhancements for qpid_qls_analyze * Store analysis and status * Recovery/reading of message content * Empty file pool status and management 5464 - [linearstore] Incompletely created journal files accumulate in EFP - 5484 1035843 Slow performance for producers - svn r.1558592 2014-01-15 fixes an issue with using /dev/random as a source of random numbers for Journal serial numbers. - svn r.1558913 2014-01-16 replaces use of /dev/urandom with several calls to rand() to construct a 64-bit random number. - * Recommend rebuilding and testing for performance again with these two fixes. Marked POST. - - 1039522 Qpid crashes while recovering from linear store around apid::linearstore::journal::JournalFile::getFqFileName() including enq_rec::decode() threw JERR_JREC_BAD_RECTAIL - * Possible dup of 1039525 - * May be fixed by QPID-5483 - waiting for needinfo, recommend rebuilding with QPID-5483 fix and re-testing. Marked POST. - - 1039525 Qpid crashes while recovering from linear store around apid::linearstore::journal::jexception::format including enq_rec::decode() threw JERR_JREC_BAD_REC_TAIL - * Possible dup of 1039522 - * May be fixed by QPID-5483 - waiting for needinfo, recommend rebuilding with QPID-5483 fix and re-testing. Marked POST. -# - 1049870 [LinearStore] auto-delete property does not survive restart +# - 1078142 [linearstore] qpidd closes connection with (distributed) transactional client while checking previous transaction, broker signals error (closed by error: Queue Ve0-2: async_dequeue() failed: exception 0x0103 wmgr::get_events() threw JERR__AIO: AIO error) + * jexception 0x0103 wmgr::get_events() threw JERR__AIO: AIO error. (AIO write operation failed: Invalid argument (-22) [pg=0 size=8192 offset=4096 fh=22]) + - 1078937 [linearstore] Installation and tests for new store analysis tool qpid-qls-analyze Fixed/closed (in commit order): =============================== @@ -85,8 +78,18 @@ NO-JIRA - Added missing Apache co svn r.1558589 2014-01-15: Proposed fix * May be linked to RHBZ 1039522 - VERIFIED * May be linked to RHBZ 1039525 - VERIFIED + - 1039522 Qpid crashes while recovering from linear store around apid::linearstore::journal::JournalFile::getFqFileName() including enq_rec::decode() threw JERR_JREC_BAD_RECTAIL + * Possible dup of 1039525 + * May be fixed by QPID-5483 - waiting for needinfo, recommend rebuilding with QPID-5483 fix and re-testing. VERIFIED. + - 1039525 Qpid crashes while recovering from linear store around apid::linearstore::journal::jexception::format including enq_rec::decode() threw JERR_JREC_BAD_REC_TAIL + * Possible dup of 1039522 + * May be fixed by QPID-5483 - waiting for needinfo, recommend rebuilding with QPID-5483 fix and re-testing. VERIFIED. 5487 1054448 [linearstore] Replace use of /dev/urandom with c random generator calls svn r.1558913 2014-01-16: Proposed fix VEFIFIED + 5484 1035843 Slow performance for producers + svn r.1558592 2014-01-15 fixes an issue with using /dev/random as a source of random numbers for Journal serial numbers. + svn r.1558913 2014-01-16 replaces use of /dev/urandom with several calls to rand() to construct a 64-bit random number. + * Recommend rebuilding and testing for performance again with these two fixes. VERIFIED. 5479 1053701 [linearstore] Using recovered store results in "JERR_JNLF_FILEOFFSOVFL: Attempted to increase submitted offset past file size. (JournalFile::submittedDblkCount)" error message * Probability: 2 of 600 (0.3%) using tx-test-soak.sh * Fixed by checkin for QPID-5480, no longer able to reproduce. VERIFIED @@ -108,6 +111,9 @@ NO-JIRA - Added missing Apache co - 1036026 [LinearStore] Qpid linear store unable to create durable queue - framing-error: Queue <q-name>: create() failed: jexception 0x0000 * UNABLE TO REPRODUCE - but Frantizek has additional info * Retested after checkin 1575009, problem solved. VERIFIED + 5651 - [C++ broker] segfault in qpid::linearstore::journal::jdir::clear_dir when declaring durable queue + svn r.1582730 2014-03-28 Proposed fix by Pavel Moravec + * Bug introduced by r.1578899. Ordered checkin list: ===================== @@ -137,6 +143,8 @@ no. svn r Q-JIRA RHBZ Date 20. 1564935 5361 - 2014-02-05 21. 1574513 5603 1063700 2014-03-05 22. 1575009 5607 1064181 2014-03-06 +23. 1578899 5362 - 2014-03-18 +24. 1582730 5651 - 2014-03-28 See above sections for details on these checkins. Modified: qpid/trunk/qpid/tools/src/py/qls/jrnl.py URL: http://svn.apache.org/viewvc/qpid/trunk/qpid/tools/src/py/qls/jrnl.py?rev=1583778&r1=1583777&r2=1583778&view=diff ============================================================================== --- qpid/trunk/qpid/tools/src/py/qls/jrnl.py (original) +++ qpid/trunk/qpid/tools/src/py/qls/jrnl.py Tue Apr 1 20:33:15 2014 @@ -63,7 +63,7 @@ class RecordHeader(object): if self.serial != file_header.serial: return False return True - def to_string(self): + def to_rh_string(self): """Return string representation of this header""" if self.is_empty(): return '0x%08x: <empty>' % (self.file_offset) @@ -80,7 +80,7 @@ class RecordHeader(object): return warn_str def __str__(self): """Return string representation of this header""" - return RecordHeader.to_string(self) + return self.to_rh_string() class RecordTail(object): FORMAT = '<4sL2Q' @@ -121,7 +121,7 @@ class RecordTail(object): return '[%c cs=0x%08x rid=0x%x]' % (magic_char, self.checksum, self.record_id) def __str__(self): """Return a string representation of the this RecordTail instance""" - return RecordTail.to_string(self) + return self.to_string() class FileHeader(RecordHeader): FORMAT = '<2H4x5QH' @@ -176,13 +176,13 @@ class FileHeader(RecordHeader): return time.strftime(fstr, now) def to_string(self): """Return a string representation of the this FileHeader instance""" - return '%s fnum=0x%x fro=0x%08x p=%d s=%dk t=%s %s' % (RecordHeader.to_string(self), self.file_num, + return '%s fnum=0x%x fro=0x%08x p=%d s=%dk t=%s %s' % (self.to_rh_string(), self.file_num, self.first_record_offset, self.partition_num, self.efp_data_size_kb, self.timestamp_str(), self._get_warnings()) def __str__(self): """Return a string representation of the this FileHeader instance""" - return FileHeader.to_string(self) + return self.to_string() class EnqueueRecord(RecordHeader): FORMAT = '<2Q' @@ -198,12 +198,12 @@ class EnqueueRecord(RecordHeader): self.data_complete = False self.record_tail = None def checksum_encode(self): # encode excluding record tail - bytes = RecordHeader.encode(self) + struct.pack(self.FORMAT, self.xid_size, self.data_size) + cs_bytes = RecordHeader.encode(self) + struct.pack(self.FORMAT, self.xid_size, self.data_size) if self.xid is not None: - bytes += self.xid + cs_bytes += self.xid if self.data is not None: - bytes += self.data - return bytes + cs_bytes += self.data + return cs_bytes def is_external(self): return self.user_flags & EnqueueRecord.EXTERNAL_FLAG_MASK > 0 def is_transient(self): @@ -253,7 +253,7 @@ class EnqueueRecord(RecordHeader): record_tail_str = '' else: record_tail_str = self.record_tail.to_string() - return '%s %s %s %s %s %s' % (RecordHeader.to_string(self), + return '%s %s %s %s %s %s' % (self.to_rh_string(), qls.utils.format_xid(self.xid, self.xid_size, show_xid_flag), qls.utils.format_data(self.data, self.data_size, show_data_flag), record_tail_str, self._print_flags(), self._get_warnings()) @@ -272,7 +272,7 @@ class EnqueueRecord(RecordHeader): return fstr def __str__(self): """Return a string representation of the this EnqueueRecord instance""" - return EnqueueRecord.to_string(self, False, False) + return self.to_string(False, False) class DequeueRecord(RecordHeader): FORMAT = '<2Q' @@ -328,7 +328,7 @@ class DequeueRecord(RecordHeader): record_tail_str = '' else: record_tail_str = self.record_tail.to_string() - return '%s drid=0x%x %s %s %s %s' % (RecordHeader.to_string(self), self.dequeue_record_id, + return '%s drid=0x%x %s %s %s %s' % (self.to_rh_string(), self.dequeue_record_id, qls.utils.format_xid(self.xid, self.xid_size, show_xid_flag), record_tail_str, self._print_flags(), self._get_warnings()) def _print_flags(self): @@ -341,7 +341,7 @@ class DequeueRecord(RecordHeader): return '' def __str__(self): """Return a string representation of the this DequeueRecord instance""" - return DequeueRecord.to_string(self, False) + return self.to_string(False) class TransactionRecord(RecordHeader): FORMAT = '<Q' @@ -387,12 +387,12 @@ class TransactionRecord(RecordHeader): record_tail_str = '' else: record_tail_str = self.record_tail.to_string() - return '%s %s %s %s' % (RecordHeader.to_string(self), + return '%s %s %s %s' % (self.to_rh_string(), qls.utils.format_xid(self.xid, self.xid_size, show_xid_flag), record_tail_str, self._get_warnings()) def __str__(self): """Return a string representation of the this TransactionRecord instance""" - return TransactionRecord.to_string(self, False) + return self.to_string(False) # ============================================================================= --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@qpid.apache.org For additional commands, e-mail: commits-h...@qpid.apache.org