[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[
https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakob Homan updated HDFS-1572:
--
Resolution: Fixed
Fix Version/s: (was: 0.21.0)
0.22.0
Assignee: Jakob Homan
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I've committed this to trunk and 22. Resolving.
Checkpointer should trigger checkpoint with specified period.
-
Key: HDFS-1572
URL: https://issues.apache.org/jira/browse/HDFS-1572
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.22.0
Reporter: Liyin Liang
Assignee: Jakob Homan
Priority: Blocker
Fix For: 0.22.0
Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572-2.patch,
HDFS-1572-3.patch, HDFS-1572.patch
{code:}
long now = now();
boolean shouldCheckpoint = false;
if(now = lastCheckpointTime + periodMSec) {
shouldCheckpoint = true;
} else {
long size = getJournalSize();
if(size = checkpointSize)
shouldCheckpoint = true;
}
{code}
{dfs.namenode.checkpoint.period} in configuration determines the period of
checkpoint. However, with above code, the Checkpointer triggers a checkpoint
every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java,
the first *if* statement should be:
{code:}
if(now = lastCheckpointTime + 1000 * checkpointPeriod) {
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[
https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakob Homan updated HDFS-1572:
--
Status: Open (was: Patch Available)
Checkpointer should trigger checkpoint with specified period.
-
Key: HDFS-1572
URL: https://issues.apache.org/jira/browse/HDFS-1572
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.22.0
Reporter: Liyin Liang
Priority: Blocker
Fix For: 0.21.0
Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572-2.patch,
HDFS-1572-3.patch, HDFS-1572.patch
{code:}
long now = now();
boolean shouldCheckpoint = false;
if(now = lastCheckpointTime + periodMSec) {
shouldCheckpoint = true;
} else {
long size = getJournalSize();
if(size = checkpointSize)
shouldCheckpoint = true;
}
{code}
{dfs.namenode.checkpoint.period} in configuration determines the period of
checkpoint. However, with above code, the Checkpointer triggers a checkpoint
every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java,
the first *if* statement should be:
{code:}
if(now = lastCheckpointTime + 1000 * checkpointPeriod) {
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[
https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakob Homan updated HDFS-1572:
--
Attachment: HDFS-1572-3.patch
Good catch Konstantin. Changed the sleep time to be the minimum of time-based
or size-based intervals. This way, regardless of whichever is lower, we'll
sleep for the minimum time needed.
Checkpointer should trigger checkpoint with specified period.
-
Key: HDFS-1572
URL: https://issues.apache.org/jira/browse/HDFS-1572
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.22.0
Reporter: Liyin Liang
Priority: Blocker
Fix For: 0.21.0
Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572-2.patch,
HDFS-1572-3.patch, HDFS-1572.patch
{code:}
long now = now();
boolean shouldCheckpoint = false;
if(now = lastCheckpointTime + periodMSec) {
shouldCheckpoint = true;
} else {
long size = getJournalSize();
if(size = checkpointSize)
shouldCheckpoint = true;
}
{code}
{dfs.namenode.checkpoint.period} in configuration determines the period of
checkpoint. However, with above code, the Checkpointer triggers a checkpoint
every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java,
the first *if* statement should be:
{code:}
if(now = lastCheckpointTime + 1000 * checkpointPeriod) {
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[
https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakob Homan updated HDFS-1572:
--
Status: Patch Available (was: Open)
Hudson!
Checkpointer should trigger checkpoint with specified period.
-
Key: HDFS-1572
URL: https://issues.apache.org/jira/browse/HDFS-1572
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.22.0
Reporter: Liyin Liang
Priority: Blocker
Fix For: 0.21.0
Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572-2.patch,
HDFS-1572-3.patch, HDFS-1572.patch
{code:}
long now = now();
boolean shouldCheckpoint = false;
if(now = lastCheckpointTime + periodMSec) {
shouldCheckpoint = true;
} else {
long size = getJournalSize();
if(size = checkpointSize)
shouldCheckpoint = true;
}
{code}
{dfs.namenode.checkpoint.period} in configuration determines the period of
checkpoint. However, with above code, the Checkpointer triggers a checkpoint
every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java,
the first *if* statement should be:
{code:}
if(now = lastCheckpointTime + 1000 * checkpointPeriod) {
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[
https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakob Homan updated HDFS-1572:
--
Attachment: HDFS-1572-2.patch
Updated patch so checkpointer only sleeps for a minute between running rather
than five. This means the checkpoint time setting will be delayed by a maximum
of a minute. Ran tests locally, all pass except known bad.
Checkpointer should trigger checkpoint with specified period.
-
Key: HDFS-1572
URL: https://issues.apache.org/jira/browse/HDFS-1572
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.22.0
Reporter: Liyin Liang
Priority: Blocker
Fix For: 0.21.0
Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572-2.patch,
HDFS-1572.patch
{code:}
long now = now();
boolean shouldCheckpoint = false;
if(now = lastCheckpointTime + periodMSec) {
shouldCheckpoint = true;
} else {
long size = getJournalSize();
if(size = checkpointSize)
shouldCheckpoint = true;
}
{code}
{dfs.namenode.checkpoint.period} in configuration determines the period of
checkpoint. However, with above code, the Checkpointer triggers a checkpoint
every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java,
the first *if* statement should be:
{code:}
if(now = lastCheckpointTime + 1000 * checkpointPeriod) {
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[
https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakob Homan updated HDFS-1572:
--
Status: Open (was: Patch Available)
Checkpointer should trigger checkpoint with specified period.
-
Key: HDFS-1572
URL: https://issues.apache.org/jira/browse/HDFS-1572
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.22.0
Reporter: Liyin Liang
Priority: Blocker
Fix For: 0.21.0
Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572-2.patch,
HDFS-1572.patch
{code:}
long now = now();
boolean shouldCheckpoint = false;
if(now = lastCheckpointTime + periodMSec) {
shouldCheckpoint = true;
} else {
long size = getJournalSize();
if(size = checkpointSize)
shouldCheckpoint = true;
}
{code}
{dfs.namenode.checkpoint.period} in configuration determines the period of
checkpoint. However, with above code, the Checkpointer triggers a checkpoint
every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java,
the first *if* statement should be:
{code:}
if(now = lastCheckpointTime + 1000 * checkpointPeriod) {
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[
https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakob Homan updated HDFS-1572:
--
Status: Patch Available (was: Open)
re-triggering hudson.
Checkpointer should trigger checkpoint with specified period.
-
Key: HDFS-1572
URL: https://issues.apache.org/jira/browse/HDFS-1572
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.22.0
Reporter: Liyin Liang
Priority: Blocker
Fix For: 0.21.0
Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572-2.patch,
HDFS-1572.patch
{code:}
long now = now();
boolean shouldCheckpoint = false;
if(now = lastCheckpointTime + periodMSec) {
shouldCheckpoint = true;
} else {
long size = getJournalSize();
if(size = checkpointSize)
shouldCheckpoint = true;
}
{code}
{dfs.namenode.checkpoint.period} in configuration determines the period of
checkpoint. However, with above code, the Checkpointer triggers a checkpoint
every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java,
the first *if* statement should be:
{code:}
if(now = lastCheckpointTime + 1000 * checkpointPeriod) {
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[
https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Liang updated HDFS-1572:
--
Attachment: 1572-2.diff
Hi Jakob,
Thanks for you work and advice. We can move some logic to static functions
to make it more testable. I'll attach a patch with a simple test case. Any
thoughts?
Checkpointer should trigger checkpoint with specified period.
-
Key: HDFS-1572
URL: https://issues.apache.org/jira/browse/HDFS-1572
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.22.0
Reporter: Liyin Liang
Priority: Blocker
Fix For: 0.21.0
Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572.patch
{code:}
long now = now();
boolean shouldCheckpoint = false;
if(now = lastCheckpointTime + periodMSec) {
shouldCheckpoint = true;
} else {
long size = getJournalSize();
if(size = checkpointSize)
shouldCheckpoint = true;
}
{code}
{dfs.namenode.checkpoint.period} in configuration determines the period of
checkpoint. However, with above code, the Checkpointer triggers a checkpoint
every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java,
the first *if* statement should be:
{code:}
if(now = lastCheckpointTime + 1000 * checkpointPeriod) {
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
[
https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Liang updated HDFS-1572:
--
Attachment: 1527-1.diff
patch for trunk.
Checkpointer should trigger checkpoint with specified period.
-
Key: HDFS-1572
URL: https://issues.apache.org/jira/browse/HDFS-1572
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Liyin Liang
Priority: Blocker
Attachments: 1527-1.diff
{code:}
long now = now();
boolean shouldCheckpoint = false;
if(now = lastCheckpointTime + periodMSec) {
shouldCheckpoint = true;
} else {
long size = getJournalSize();
if(size = checkpointSize)
shouldCheckpoint = true;
}
{code}
{dfs.namenode.checkpoint.period} in configuration determines the period of
checkpoint. However, with above code, the Checkpointer triggers a checkpoint
every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java,
the first *if* statement should be:
{code:}
if(now = lastCheckpointTime + 1000 * checkpointPeriod) {
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
