[jira] [Commented] (HDFS-3203) Currently，The Checkpointer is controled by time.in this way,it must be checkponit in that it is only one transaction in checkpoint period.I think it need add file size t
[ https://issues.apache.org/jira/browse/HDFS-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248039#comment-13248039 ] liaowenrui commented on HDFS-3203: -- thanks,Todd, my mistake. you are right, I will close this issue. > Currently，The Checkpointer is controled by time.in this way,it must be > checkponit in that it is only one transaction in checkpoint period.I think it > need add file size to control checkpoint > -- > > Key: HDFS-3203 > URL: https://issues.apache.org/jira/browse/HDFS-3203 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.24.0, 2.0.0 >Reporter: liaowenrui > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3203) Currently，The Checkpointer is controled by time.in this way,it must be checkponit in that it is only one transaction in checkpoint period.I think it need add file size t
[
https://issues.apache.org/jira/browse/HDFS-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248023#comment-13248023
]
Todd Lipcon commented on HDFS-3203:
---
bq. 1.dfs.namenode.checkpoint.txns is not used in ha
It is supposed to be used. See this code:
{code}
if (uncheckpointed >= checkpointConf.getTxnCount()) {
LOG.info("Triggering checkpoint because there have been " +
uncheckpointed + " txns since the last checkpoint, which " +
"exceeds the configured threshold " +
checkpointConf.getTxnCount());
needCheckpoint = true;
} else if (secsSinceLast >= checkpointConf.getPeriod()) {
LOG.info("Triggering checkpoint because it has been " +
secsSinceLast + " seconds since the last checkpoint, which " +
"exceeds the configured interval " +
checkpointConf.getPeriod());
needCheckpoint = true;
}
{code}
If it is not working, please explain how to reproduce.
bq. 2.why standbyCheckpointer is running in active namenode and standby
namenode?
The daemon only runs when the node is in standby mode. When it becomes active,
it stops the checkpointer.
> Currently，The Checkpointer is controled by time.in this way,it must be
> checkponit in that it is only one transaction in checkpoint period.I think it
> need add file size to control checkpoint
> --
>
> Key: HDFS-3203
> URL: https://issues.apache.org/jira/browse/HDFS-3203
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
>Affects Versions: 0.24.0, 2.0.0
>Reporter: liaowenrui
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3203) Currently，The Checkpointer is controled by time.in this way,it must be checkponit in that it is only one transaction in checkpoint period.I think it need add file size t
[ https://issues.apache.org/jira/browse/HDFS-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248001#comment-13248001 ] liaowenrui commented on HDFS-3203: -- I mean that checkpointer is not start in ha,also dfs.namenode.checkpoint.txns is not become effective. standbyCheckpointer is running in the ha,and it is running in active namenode and stanby namenode. so I have two doubt: 1.dfs.namenode.checkpoint.txns is not used in ha 2.why standbyCheckpointer is running in active namenode and standby namenode? > Currently，The Checkpointer is controled by time.in this way,it must be > checkponit in that it is only one transaction in checkpoint period.I think it > need add file size to control checkpoint > -- > > Key: HDFS-3203 > URL: https://issues.apache.org/jira/browse/HDFS-3203 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.24.0, 2.0.0 >Reporter: liaowenrui > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3203) Currently，The Checkpointer is controled by time.in this way,it must be checkponit in that it is only one transaction in checkpoint period.I think it need add file size t
[ https://issues.apache.org/jira/browse/HDFS-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247441#comment-13247441 ] Todd Lipcon commented on HDFS-3203: --- I'm sorry, I don't understand your question. Can you please clarify? > Currently，The Checkpointer is controled by time.in this way,it must be > checkponit in that it is only one transaction in checkpoint period.I think it > need add file size to control checkpoint > -- > > Key: HDFS-3203 > URL: https://issues.apache.org/jira/browse/HDFS-3203 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.24.0, 2.0.0 >Reporter: liaowenrui > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3203) Currently，The Checkpointer is controled by time.in this way,it must be checkponit in that it is only one transaction in checkpoint period.I think it need add file size t
[ https://issues.apache.org/jira/browse/HDFS-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247128#comment-13247128 ] liaowenrui commented on HDFS-3203: -- thank you,Todd, in nomal the editlog size is controled by dfs.namenode.checkpoint.txns. but,you see fllow scenario: I have a doubt,why it don't start checkponter in ha? why standbyCheckpointer is running in active namenode and standby namenode? > Currently，The Checkpointer is controled by time.in this way,it must be > checkponit in that it is only one transaction in checkpoint period.I think it > need add file size to control checkpoint > -- > > Key: HDFS-3203 > URL: https://issues.apache.org/jira/browse/HDFS-3203 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.24.0, 2.0.0 >Reporter: liaowenrui > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3203) Currently，The Checkpointer is controled by time.in this way,it must be checkponit in that it is only one transaction in checkpoint period.I think it need add file size t
[ https://issues.apache.org/jira/browse/HDFS-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247068#comment-13247068 ] Todd Lipcon commented on HDFS-3203: --- You can configure dfs.namenode.checkpoint.txns to the desired number of transactoins, and then set dfs.namenode.checkpoint.period to a very high value. This will give you the desired behavior. Does that not satisfy your requirements? > Currently，The Checkpointer is controled by time.in this way,it must be > checkponit in that it is only one transaction in checkpoint period.I think it > need add file size to control checkpoint > -- > > Key: HDFS-3203 > URL: https://issues.apache.org/jira/browse/HDFS-3203 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.24.0, 2.0.0 >Reporter: liaowenrui > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3203) Currently，The Checkpointer is controled by time.in this way,it must be checkponit in that it is only one transaction in checkpoint period.I think it need add file size t
[ https://issues.apache.org/jira/browse/HDFS-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247057#comment-13247057 ] liaowenrui commented on HDFS-3203: -- only controled by time,the edits file is very small,when the system is idlesse.in this case,it produce a good many of edits file. -rw-r--r-- 1 root root 30 Apr 4 20:54 edits_001-002 -rw-r--r-- 1 root root 1.7M Apr 4 21:54 edits_003-0017336 -rw-r--r-- 1 root root 1.1M Apr 4 22:54 edits_0017337-0026001 -rw-r--r-- 1 root root 30 Apr 4 23:54 edits_0026002-0026003 -rw-r--r-- 1 root root 232 Apr 5 00:54 edits_0026004-0026007 -rw-r--r-- 1 root root 160 Apr 5 01:54 edits_0026008-0026010 -rw-r--r-- 1 root root 30 Apr 5 02:54 edits_0026011-0026012 -rw-r--r-- 1 root root 30 Apr 5 03:54 edits_0026013-0026014 -rw-r--r-- 1 root root 30 Apr 5 04:54 edits_0026015-0026016 -rw-r--r-- 1 root root 30 Apr 5 05:54 edits_0026017-0026018 -rw-r--r-- 1 root root 30 Apr 5 06:54 edits_0026019-0026020 -rw-r--r-- 1 root root 30 Apr 5 07:54 edits_0026021-0026022 -rw-r--r-- 1 root root 30 Apr 5 08:54 edits_0026023-0026024 -rw-r--r-- 1 root root 1.7M Apr 5 09:54 edits_0026025-0043334 -rw-r--r-- 1 root root 2.2K Apr 5 10:54 edits_0043335-0043358 > Currently，The Checkpointer is controled by time.in this way,it must be > checkponit in that it is only one transaction in checkpoint period.I think it > need add file size to control checkpoint > -- > > Key: HDFS-3203 > URL: https://issues.apache.org/jira/browse/HDFS-3203 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.24.0, 2.0.0 >Reporter: liaowenrui > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3203) Currently，The Checkpointer is controled by time.in this way,it must be checkponit in that it is only one transaction in checkpoint period.I think it need add file size t
[ https://issues.apache.org/jira/browse/HDFS-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247056#comment-13247056 ] liaowenrui commented on HDFS-3203: -- only controled by time,the edits file is very small,when the system is idlesse.in this case,it produce a good many of edits file. -rw-r--r-- 1 root root 30 Apr 4 20:54 edits_001-002 -rw-r--r-- 1 root root 1.7M Apr 4 21:54 edits_003-0017336 -rw-r--r-- 1 root root 1.1M Apr 4 22:54 edits_0017337-0026001 -rw-r--r-- 1 root root 30 Apr 4 23:54 edits_0026002-0026003 -rw-r--r-- 1 root root 232 Apr 5 00:54 edits_0026004-0026007 -rw-r--r-- 1 root root 160 Apr 5 01:54 edits_0026008-0026010 -rw-r--r-- 1 root root 30 Apr 5 02:54 edits_0026011-0026012 -rw-r--r-- 1 root root 30 Apr 5 03:54 edits_0026013-0026014 -rw-r--r-- 1 root root 30 Apr 5 04:54 edits_0026015-0026016 -rw-r--r-- 1 root root 30 Apr 5 05:54 edits_0026017-0026018 -rw-r--r-- 1 root root 30 Apr 5 06:54 edits_0026019-0026020 -rw-r--r-- 1 root root 30 Apr 5 07:54 edits_0026021-0026022 -rw-r--r-- 1 root root 30 Apr 5 08:54 edits_0026023-0026024 -rw-r--r-- 1 root root 1.7M Apr 5 09:54 edits_0026025-0043334 -rw-r--r-- 1 root root 2.2K Apr 5 10:54 edits_0043335-0043358 > Currently，The Checkpointer is controled by time.in this way,it must be > checkponit in that it is only one transaction in checkpoint period.I think it > need add file size to control checkpoint > -- > > Key: HDFS-3203 > URL: https://issues.apache.org/jira/browse/HDFS-3203 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.24.0, 2.0.0 >Reporter: liaowenrui > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
