[
https://issues.apache.org/jira/browse/HADOOP-14604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dingguotao updated HADOOP-14604:
--------------------------------
Description:
With HDFS Federation configured, the aggregation log have a default replication
count of 1, rather than the value of dfs.replication in hdfs-site.xml
configuration file.
Before:
{code:java}
[op@NM-304-SA5212M4-BIGDATA-640 ~]$ hdfs dfs -ls
/yarn/apps/op/logs/application_1498544648312_0001
Found 3 items
-rw-r----- 1 op hadoop 8270 2017-06-27 14:25
/yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-645_8041
-rw-r----- 1 op hadoop 54469 2017-06-27 14:25
/yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-646_8041
-rw-r----- 1 op hadoop 6537 2017-06-27 14:25
/yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-672_8041
{code}
The aggregation log have only 1 replication in default. But the value of
dfs.replication in hdfs-site.xml configuration file is 3
After apply this patch:
{code:java}
[op@NM-304-SA5212M4-BIGDATA-640 ~]$ hdfs dfs -ls
/yarn/apps/op/logs/application_1498635214020_0001
Found 3 items
-rw-r----- 3 op hadoop 5729 2017-06-28 15:34
/yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-645_8041
-rw-r----- 3 op hadoop 38439 2017-06-28 15:34
/yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-646_8041
-rw-r----- 3 op hadoop 8270 2017-06-28 15:34
/yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-671_8041
{code}
was:
With HDFS Federation configured, the aggregation log have a default replication
count of 1, rather than the value of dfs.replication in hdfs-site.xml
configuration file.
Before:
{code:java}
[op@NM-304-SA5212M4-BIGDATA-640 ~]$ hdfs dfs -ls
/yarn/apps/op/logs/application_1498544648312_0001
Found 3 items
-rw-r----- 1 op hadoop 8270 2017-06-27 14:25
/yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-645_8041
-rw-r----- 1 op hadoop 54469 2017-06-27 14:25
/yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-646_8041
-rw-r----- 1 op hadoop 6537 2017-06-27 14:25
/yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-672_8041
{code}
The aggregation log have only 1 replication in default. But the value of
dfs.replication in hdfs-site.xml configuration file is 3
After apply this patch:
{code:java}
[op@NM-304-SA5212M4-BIGDATA-640 ~]$ hdfs dfs -ls
/yarn/apps/op/logs/application_1498635214020_0001
Found 3 items
-rw-r----- 3 op hadoop 5729 2017-06-28 15:34
/yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-645_8041
-rw-r----- 3 op hadoop 38439 2017-06-28 15:34
/yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-646_8041
-rw-r----- 3 op hadoop 8270 2017-06-28 15:34
/yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-671_8041
{code}
> Fix the bug of aggregation log have one replication in HDFS Federation
> environment
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-14604
> URL: https://issues.apache.org/jira/browse/HADOOP-14604
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
> Affects Versions: 2.6.0
> Environment: Hadoop 2.6.0 with HDFS Federation
> Reporter: dingguotao
> Labels: patch
> Attachments: HADOOP-14604.patch
>
>
> With HDFS Federation configured, the aggregation log have a default
> replication count of 1, rather than the value of dfs.replication in
> hdfs-site.xml configuration file.
> Before:
> {code:java}
> [op@NM-304-SA5212M4-BIGDATA-640 ~]$ hdfs dfs -ls
> /yarn/apps/op/logs/application_1498544648312_0001
> Found 3 items
> -rw-r----- 1 op hadoop 8270 2017-06-27 14:25
> /yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-645_8041
> -rw-r----- 1 op hadoop 54469 2017-06-27 14:25
> /yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-646_8041
> -rw-r----- 1 op hadoop 6537 2017-06-27 14:25
> /yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-672_8041
> {code}
> The aggregation log have only 1 replication in default. But the value of
> dfs.replication in hdfs-site.xml configuration file is 3
> After apply this patch:
> {code:java}
> [op@NM-304-SA5212M4-BIGDATA-640 ~]$ hdfs dfs -ls
> /yarn/apps/op/logs/application_1498635214020_0001
> Found 3 items
> -rw-r----- 3 op hadoop 5729 2017-06-28 15:34
> /yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-645_8041
> -rw-r----- 3 op hadoop 38439 2017-06-28 15:34
> /yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-646_8041
> -rw-r----- 3 op hadoop 8270 2017-06-28 15:34
> /yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-671_8041
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
