[jira] [Commented] (AMQ-6936) Dead loop on log file reading
[ https://issues.apache.org/jira/browse/AMQ-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16411151#comment-16411151 ] Rural Hunter commented on AMQ-6936: --- OK, that's nice, thanks! > Dead loop on log file reading > - > > Key: AMQ-6936 > URL: https://issues.apache.org/jira/browse/AMQ-6936 > Project: ActiveMQ > Issue Type: Bug > Components: KahaDB >Affects Versions: 5.13.4 >Reporter: Rural Hunter >Priority: Critical > > We restarted our activemq instance(5.13.4) but found it hung on recovering > data. I got the data and ran a debug and found these: > # The dead loop is in Journal.getNextLocation(Location location) > # I added log in the method and this is I got: > 2018-03-23 14:56:43 [DEBUG] Journal Get next location for: 7:33554239 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554330 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554330: offset=33554330, > size=28, type=2 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554358 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554358: offset=33554358, > size=91, type=1 > 2018-03-23 14:56:43 [DEBUG] Journal Get next location for: 7:33554358 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554449 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554449: offset=33554449, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > # 33554432 is the max log file size. activemq read a location at the end of > the file and the size is 0, this causes the dead loop > I don't know if this is a problem when the file is saved or there is > something need to be fixed when reading. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-6936) Dead loop on log file reading
[ https://issues.apache.org/jira/browse/AMQ-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16411134#comment-16411134 ] Gary Tully commented on AMQ-6936: - That was resolved via: https://git-wip-us.apache.org/repos/asf?p=activemq.git;a=blobdiff;f=activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/disk/journal/Journal.java;h=182a3d7757541627d67b8bf4f539836dc259bc0b;hp=da0d5b432d2eb0783b7eee686a2cb1c087124f85;hb=62bdbb0;hpb=3c342ffce43546667f59bb16db373061724e1b0f https://issues.apache.org/jira/browse/AMQ-5603 > Dead loop on log file reading > - > > Key: AMQ-6936 > URL: https://issues.apache.org/jira/browse/AMQ-6936 > Project: ActiveMQ > Issue Type: Bug > Components: KahaDB >Affects Versions: 5.13.4 >Reporter: Rural Hunter >Priority: Critical > > We restarted our activemq instance(5.13.4) but found it hung on recovering > data. I got the data and ran a debug and found these: > # The dead loop is in Journal.getNextLocation(Location location) > # I added log in the method and this is I got: > 2018-03-23 14:56:43 [DEBUG] Journal Get next location for: 7:33554239 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554330 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554330: offset=33554330, > size=28, type=2 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554358 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554358: offset=33554358, > size=91, type=1 > 2018-03-23 14:56:43 [DEBUG] Journal Get next location for: 7:33554358 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554449 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554449: offset=33554449, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > # 33554432 is the max log file size. activemq read a location at the end of > the file and the size is 0, this causes the dead loop > I don't know if this is a problem when the file is saved or there is > something need to be fixed when reading. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-6936) Dead loop on log file reading
[ https://issues.apache.org/jira/browse/AMQ-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16411049#comment-16411049 ] Rural Hunter commented on AMQ-6936: --- I'm not sure if it is allowed that the file size is greater than the maxFileLength. If it is allowed, then this might should be changed: else if (cur.getType() == 0) { // eof - jump to next datafile cur.setOffset(maxFileLength); => change to: cur.setOffset(dataFile.getLength()); } With the change above, my activemq instance can start up. > Dead loop on log file reading > - > > Key: AMQ-6936 > URL: https://issues.apache.org/jira/browse/AMQ-6936 > Project: ActiveMQ > Issue Type: Bug > Components: KahaDB >Affects Versions: 5.13.4 >Reporter: Rural Hunter >Priority: Critical > > We restarted our activemq instance(5.13.4) but found it hung on recovering > data. I got the data and ran a debug and found these: > # The dead loop is in Journal.getNextLocation(Location location) > # I added log in the method and this is I got: > 2018-03-23 14:56:43 [DEBUG] Journal Get next location for: 7:33554239 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554330 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554330: offset=33554330, > size=28, type=2 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554358 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554358: offset=33554358, > size=91, type=1 > 2018-03-23 14:56:43 [DEBUG] Journal Get next location for: 7:33554358 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554449 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554449: offset=33554449, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > # 33554432 is the max log file size. activemq read a location at the end of > the file and the size is 0, this causes the dead loop > I don't know if this is a problem when the file is saved or there is > something need to be fixed when reading. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AMQ-6936) Dead loop on log file reading
[ https://issues.apache.org/jira/browse/AMQ-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16411023#comment-16411023 ] Rural Hunter commented on AMQ-6936: --- These are the data files: -rw-rw-r-- 1 activemq activemq 33554432 3月 23 11:46 db-13.log -rw-rw-r-- 1 activemq activemq 33554432 3月 23 11:46 db-1.log -rw-rw-r-- 1 activemq activemq 33554432 3月 23 11:46 db-2.log -rw-rw-r-- 1 activemq activemq 33554432 3月 23 11:46 db-4.log -rw-rw-r-- 1 activemq activemq 33554432 3月 23 11:46 db-5.log -rw-rw-r-- 1 activemq activemq 33554454 3月 23 11:46 db-7.log -rw-rw-r-- 1 activemq activemq 32768 3月 23 12:18 db.data -rw-rw-r-- 1 activemq activemq 28720 3月 23 11:46 db.redo db-7.log is larger than the maxFileLength = DEFAULT_MAX_FILE_LENGTH=33554432 > Dead loop on log file reading > - > > Key: AMQ-6936 > URL: https://issues.apache.org/jira/browse/AMQ-6936 > Project: ActiveMQ > Issue Type: Bug > Components: KahaDB >Affects Versions: 5.13.4 >Reporter: Rural Hunter >Priority: Critical > > We restarted our activemq instance(5.13.4) but found it hung on recovering > data. I got the data and ran a debug and found these: > # The dead loop is in Journal.getNextLocation(Location location) > # I added log in the method and this is I got: > 2018-03-23 14:56:43 [DEBUG] Journal Get next location for: 7:33554239 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554330 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554330: offset=33554330, > size=28, type=2 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554358 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554358: offset=33554358, > size=91, type=1 > 2018-03-23 14:56:43 [DEBUG] Journal Get next location for: 7:33554358 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554449 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554449: offset=33554449, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > 2018-03-23 14:56:43 [DEBUG] Journal Reading location: 7:33554432 > 2018-03-23 14:56:43 [DEBUG] Journal Location 7:33554432: offset=33554432, > size=0, type=0 > # 33554432 is the max log file size. activemq read a location at the end of > the file and the size is 0, this causes the dead loop > I don't know if this is a problem when the file is saved or there is > something need to be fixed when reading. -- This message was sent by Atlassian JIRA (v7.6.3#76005)