[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[ https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-1572: -- Resolution: Fixed Fix Version/s: (was: 0.21.0) 0.22.0 Assignee: Jakob Homan Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) I've committed this to trunk and 22. Resolving. Checkpointer should trigger checkpoint with specified period. - Key: HDFS-1572 URL: https://issues.apache.org/jira/browse/HDFS-1572 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.22.0 Reporter: Liyin Liang Assignee: Jakob Homan Priority: Blocker Fix For: 0.22.0 Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572-2.patch, HDFS-1572-3.patch, HDFS-1572.patch {code:} long now = now(); boolean shouldCheckpoint = false; if(now = lastCheckpointTime + periodMSec) { shouldCheckpoint = true; } else { long size = getJournalSize(); if(size = checkpointSize) shouldCheckpoint = true; } {code} {dfs.namenode.checkpoint.period} in configuration determines the period of checkpoint. However, with above code, the Checkpointer triggers a checkpoint every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java, the first *if* statement should be: {code:} if(now = lastCheckpointTime + 1000 * checkpointPeriod) { {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[ https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-1572: -- Status: Open (was: Patch Available) Checkpointer should trigger checkpoint with specified period. - Key: HDFS-1572 URL: https://issues.apache.org/jira/browse/HDFS-1572 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.22.0 Reporter: Liyin Liang Priority: Blocker Fix For: 0.21.0 Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572-2.patch, HDFS-1572-3.patch, HDFS-1572.patch {code:} long now = now(); boolean shouldCheckpoint = false; if(now = lastCheckpointTime + periodMSec) { shouldCheckpoint = true; } else { long size = getJournalSize(); if(size = checkpointSize) shouldCheckpoint = true; } {code} {dfs.namenode.checkpoint.period} in configuration determines the period of checkpoint. However, with above code, the Checkpointer triggers a checkpoint every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java, the first *if* statement should be: {code:} if(now = lastCheckpointTime + 1000 * checkpointPeriod) { {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[ https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-1572: -- Attachment: HDFS-1572-3.patch Good catch Konstantin. Changed the sleep time to be the minimum of time-based or size-based intervals. This way, regardless of whichever is lower, we'll sleep for the minimum time needed. Checkpointer should trigger checkpoint with specified period. - Key: HDFS-1572 URL: https://issues.apache.org/jira/browse/HDFS-1572 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.22.0 Reporter: Liyin Liang Priority: Blocker Fix For: 0.21.0 Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572-2.patch, HDFS-1572-3.patch, HDFS-1572.patch {code:} long now = now(); boolean shouldCheckpoint = false; if(now = lastCheckpointTime + periodMSec) { shouldCheckpoint = true; } else { long size = getJournalSize(); if(size = checkpointSize) shouldCheckpoint = true; } {code} {dfs.namenode.checkpoint.period} in configuration determines the period of checkpoint. However, with above code, the Checkpointer triggers a checkpoint every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java, the first *if* statement should be: {code:} if(now = lastCheckpointTime + 1000 * checkpointPeriod) { {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[ https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-1572: -- Status: Patch Available (was: Open) Hudson! Checkpointer should trigger checkpoint with specified period. - Key: HDFS-1572 URL: https://issues.apache.org/jira/browse/HDFS-1572 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.22.0 Reporter: Liyin Liang Priority: Blocker Fix For: 0.21.0 Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572-2.patch, HDFS-1572-3.patch, HDFS-1572.patch {code:} long now = now(); boolean shouldCheckpoint = false; if(now = lastCheckpointTime + periodMSec) { shouldCheckpoint = true; } else { long size = getJournalSize(); if(size = checkpointSize) shouldCheckpoint = true; } {code} {dfs.namenode.checkpoint.period} in configuration determines the period of checkpoint. However, with above code, the Checkpointer triggers a checkpoint every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java, the first *if* statement should be: {code:} if(now = lastCheckpointTime + 1000 * checkpointPeriod) { {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[ https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-1572: -- Attachment: HDFS-1572-2.patch Updated patch so checkpointer only sleeps for a minute between running rather than five. This means the checkpoint time setting will be delayed by a maximum of a minute. Ran tests locally, all pass except known bad. Checkpointer should trigger checkpoint with specified period. - Key: HDFS-1572 URL: https://issues.apache.org/jira/browse/HDFS-1572 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.22.0 Reporter: Liyin Liang Priority: Blocker Fix For: 0.21.0 Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572-2.patch, HDFS-1572.patch {code:} long now = now(); boolean shouldCheckpoint = false; if(now = lastCheckpointTime + periodMSec) { shouldCheckpoint = true; } else { long size = getJournalSize(); if(size = checkpointSize) shouldCheckpoint = true; } {code} {dfs.namenode.checkpoint.period} in configuration determines the period of checkpoint. However, with above code, the Checkpointer triggers a checkpoint every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java, the first *if* statement should be: {code:} if(now = lastCheckpointTime + 1000 * checkpointPeriod) { {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[ https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-1572: -- Status: Open (was: Patch Available) Checkpointer should trigger checkpoint with specified period. - Key: HDFS-1572 URL: https://issues.apache.org/jira/browse/HDFS-1572 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.22.0 Reporter: Liyin Liang Priority: Blocker Fix For: 0.21.0 Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572-2.patch, HDFS-1572.patch {code:} long now = now(); boolean shouldCheckpoint = false; if(now = lastCheckpointTime + periodMSec) { shouldCheckpoint = true; } else { long size = getJournalSize(); if(size = checkpointSize) shouldCheckpoint = true; } {code} {dfs.namenode.checkpoint.period} in configuration determines the period of checkpoint. However, with above code, the Checkpointer triggers a checkpoint every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java, the first *if* statement should be: {code:} if(now = lastCheckpointTime + 1000 * checkpointPeriod) { {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[ https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-1572: -- Status: Patch Available (was: Open) re-triggering hudson. Checkpointer should trigger checkpoint with specified period. - Key: HDFS-1572 URL: https://issues.apache.org/jira/browse/HDFS-1572 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.22.0 Reporter: Liyin Liang Priority: Blocker Fix For: 0.21.0 Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572-2.patch, HDFS-1572.patch {code:} long now = now(); boolean shouldCheckpoint = false; if(now = lastCheckpointTime + periodMSec) { shouldCheckpoint = true; } else { long size = getJournalSize(); if(size = checkpointSize) shouldCheckpoint = true; } {code} {dfs.namenode.checkpoint.period} in configuration determines the period of checkpoint. However, with above code, the Checkpointer triggers a checkpoint every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java, the first *if* statement should be: {code:} if(now = lastCheckpointTime + 1000 * checkpointPeriod) { {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[ https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Liang updated HDFS-1572: -- Attachment: 1572-2.diff Hi Jakob, Thanks for you work and advice. We can move some logic to static functions to make it more testable. I'll attach a patch with a simple test case. Any thoughts? Checkpointer should trigger checkpoint with specified period. - Key: HDFS-1572 URL: https://issues.apache.org/jira/browse/HDFS-1572 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.22.0 Reporter: Liyin Liang Priority: Blocker Fix For: 0.21.0 Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572.patch {code:} long now = now(); boolean shouldCheckpoint = false; if(now = lastCheckpointTime + periodMSec) { shouldCheckpoint = true; } else { long size = getJournalSize(); if(size = checkpointSize) shouldCheckpoint = true; } {code} {dfs.namenode.checkpoint.period} in configuration determines the period of checkpoint. However, with above code, the Checkpointer triggers a checkpoint every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java, the first *if* statement should be: {code:} if(now = lastCheckpointTime + 1000 * checkpointPeriod) { {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[ https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Liang updated HDFS-1572: -- Attachment: 1527-1.diff patch for trunk. Checkpointer should trigger checkpoint with specified period. - Key: HDFS-1572 URL: https://issues.apache.org/jira/browse/HDFS-1572 Project: Hadoop HDFS Issue Type: Bug Reporter: Liyin Liang Priority: Blocker Attachments: 1527-1.diff {code:} long now = now(); boolean shouldCheckpoint = false; if(now = lastCheckpointTime + periodMSec) { shouldCheckpoint = true; } else { long size = getJournalSize(); if(size = checkpointSize) shouldCheckpoint = true; } {code} {dfs.namenode.checkpoint.period} in configuration determines the period of checkpoint. However, with above code, the Checkpointer triggers a checkpoint every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java, the first *if* statement should be: {code:} if(now = lastCheckpointTime + 1000 * checkpointPeriod) { {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.