[jira] [Commented] (HDFS-9027) Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
[
https://issues.apache.org/jira/browse/HDFS-9027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741391#comment-14741391
]
Hudson commented on HDFS-9027:
--
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #379 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/379/])
HDFS-9027. Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method.
(Contributed by Mingliang Liu) (arp: rev
15a557fcfec5eceedde9f1597385d5d3b01b2cd7)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java
*
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
> Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
> ---
>
> Key: HDFS-9027
> URL: https://issues.apache.org/jira/browse/HDFS-9027
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HDFS-9027.000.patch, HDFS-9027.001.patch
>
>
> In method {{isLazyPersist()}}, the {{org.apache.hadoop.hdfs.DataStreamer}}
> class checks whether the HDFS file is lazy persist. It does two things:
> 1. Create a class-wide _static_ {{BlockStoragePolicySuite}} object, which
> builds an array of {{BlockStoragePolicy}} internally
> 2. Get a block storage policy object from the {{blockStoragePolicySuite}} by
> policy name {{HdfsConstants.MEMORY_STORAGE_POLICY_NAME}}
> This has two side effects:
> 1. Takes time to iterate the pre-built block storage policy array in order to
> find the _same_ policy every time whose id matters only (as we need to
> compare the file status policy id with lazy persist policy id)
> 2. {{DataStreamer}} class imports {{BlockStoragePolicySuite}}. The former
> should be moved to {{hadoop-hdfs-client}} module, while the latter can stay
> in {{hadoop-hdfs}} module.
> Actually, we have the block storage policy IDs, which can be used to compare
> with HDFS file status' policy id, as following:
> {code}
> static boolean isLazyPersist(HdfsFileStatus stat) {
> return stat.getStoragePolicy() == HdfsConstants.MEMORY_STORAGE_POLICY_ID;
> }
> {code}
> This way, we only need to move the block storage policies' IDs from
> {{HdfsServerConstant}} ({{hadoop-hdfs}} module) to {{HdfsConstants}}
> ({{hadoop-hdfs-client}} module).
> Another reason we should move those block storage policy IDs is that the
> block storage policy names were moved to {{HdfsConstants}} already.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9027) Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
[
https://issues.apache.org/jira/browse/HDFS-9027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741436#comment-14741436
]
Hudson commented on HDFS-9027:
--
FAILURE: Integrated in Hadoop-Yarn-trunk # (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk//])
HDFS-9027. Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method.
(Contributed by Mingliang Liu) (arp: rev
15a557fcfec5eceedde9f1597385d5d3b01b2cd7)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
*
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java
> Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
> ---
>
> Key: HDFS-9027
> URL: https://issues.apache.org/jira/browse/HDFS-9027
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HDFS-9027.000.patch, HDFS-9027.001.patch
>
>
> In method {{isLazyPersist()}}, the {{org.apache.hadoop.hdfs.DataStreamer}}
> class checks whether the HDFS file is lazy persist. It does two things:
> 1. Create a class-wide _static_ {{BlockStoragePolicySuite}} object, which
> builds an array of {{BlockStoragePolicy}} internally
> 2. Get a block storage policy object from the {{blockStoragePolicySuite}} by
> policy name {{HdfsConstants.MEMORY_STORAGE_POLICY_NAME}}
> This has two side effects:
> 1. Takes time to iterate the pre-built block storage policy array in order to
> find the _same_ policy every time whose id matters only (as we need to
> compare the file status policy id with lazy persist policy id)
> 2. {{DataStreamer}} class imports {{BlockStoragePolicySuite}}. The former
> should be moved to {{hadoop-hdfs-client}} module, while the latter can stay
> in {{hadoop-hdfs}} module.
> Actually, we have the block storage policy IDs, which can be used to compare
> with HDFS file status' policy id, as following:
> {code}
> static boolean isLazyPersist(HdfsFileStatus stat) {
> return stat.getStoragePolicy() == HdfsConstants.MEMORY_STORAGE_POLICY_ID;
> }
> {code}
> This way, we only need to move the block storage policies' IDs from
> {{HdfsServerConstant}} ({{hadoop-hdfs}} module) to {{HdfsConstants}}
> ({{hadoop-hdfs-client}} module).
> Another reason we should move those block storage policy IDs is that the
> block storage policy names were moved to {{HdfsConstants}} already.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9027) Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
[
https://issues.apache.org/jira/browse/HDFS-9027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741443#comment-14741443
]
Hudson commented on HDFS-9027:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #373 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/373/])
HDFS-9027. Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method.
(Contributed by Mingliang Liu) (arp: rev
15a557fcfec5eceedde9f1597385d5d3b01b2cd7)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
*
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java
> Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
> ---
>
> Key: HDFS-9027
> URL: https://issues.apache.org/jira/browse/HDFS-9027
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HDFS-9027.000.patch, HDFS-9027.001.patch
>
>
> In method {{isLazyPersist()}}, the {{org.apache.hadoop.hdfs.DataStreamer}}
> class checks whether the HDFS file is lazy persist. It does two things:
> 1. Create a class-wide _static_ {{BlockStoragePolicySuite}} object, which
> builds an array of {{BlockStoragePolicy}} internally
> 2. Get a block storage policy object from the {{blockStoragePolicySuite}} by
> policy name {{HdfsConstants.MEMORY_STORAGE_POLICY_NAME}}
> This has two side effects:
> 1. Takes time to iterate the pre-built block storage policy array in order to
> find the _same_ policy every time whose id matters only (as we need to
> compare the file status policy id with lazy persist policy id)
> 2. {{DataStreamer}} class imports {{BlockStoragePolicySuite}}. The former
> should be moved to {{hadoop-hdfs-client}} module, while the latter can stay
> in {{hadoop-hdfs}} module.
> Actually, we have the block storage policy IDs, which can be used to compare
> with HDFS file status' policy id, as following:
> {code}
> static boolean isLazyPersist(HdfsFileStatus stat) {
> return stat.getStoragePolicy() == HdfsConstants.MEMORY_STORAGE_POLICY_ID;
> }
> {code}
> This way, we only need to move the block storage policies' IDs from
> {{HdfsServerConstant}} ({{hadoop-hdfs}} module) to {{HdfsConstants}}
> ({{hadoop-hdfs-client}} module).
> Another reason we should move those block storage policy IDs is that the
> block storage policy names were moved to {{HdfsConstants}} already.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9027) Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
[
https://issues.apache.org/jira/browse/HDFS-9027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741539#comment-14741539
]
Hudson commented on HDFS-9027:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #2298 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2298/])
HDFS-9027. Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method.
(Contributed by Mingliang Liu) (arp: rev
15a557fcfec5eceedde9f1597385d5d3b01b2cd7)
*
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
> Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
> ---
>
> Key: HDFS-9027
> URL: https://issues.apache.org/jira/browse/HDFS-9027
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HDFS-9027.000.patch, HDFS-9027.001.patch
>
>
> In method {{isLazyPersist()}}, the {{org.apache.hadoop.hdfs.DataStreamer}}
> class checks whether the HDFS file is lazy persist. It does two things:
> 1. Create a class-wide _static_ {{BlockStoragePolicySuite}} object, which
> builds an array of {{BlockStoragePolicy}} internally
> 2. Get a block storage policy object from the {{blockStoragePolicySuite}} by
> policy name {{HdfsConstants.MEMORY_STORAGE_POLICY_NAME}}
> This has two side effects:
> 1. Takes time to iterate the pre-built block storage policy array in order to
> find the _same_ policy every time whose id matters only (as we need to
> compare the file status policy id with lazy persist policy id)
> 2. {{DataStreamer}} class imports {{BlockStoragePolicySuite}}. The former
> should be moved to {{hadoop-hdfs-client}} module, while the latter can stay
> in {{hadoop-hdfs}} module.
> Actually, we have the block storage policy IDs, which can be used to compare
> with HDFS file status' policy id, as following:
> {code}
> static boolean isLazyPersist(HdfsFileStatus stat) {
> return stat.getStoragePolicy() == HdfsConstants.MEMORY_STORAGE_POLICY_ID;
> }
> {code}
> This way, we only need to move the block storage policies' IDs from
> {{HdfsServerConstant}} ({{hadoop-hdfs}} module) to {{HdfsConstants}}
> ({{hadoop-hdfs-client}} module).
> Another reason we should move those block storage policy IDs is that the
> block storage policy names were moved to {{HdfsConstants}} already.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9027) Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
[
https://issues.apache.org/jira/browse/HDFS-9027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741552#comment-14741552
]
Hudson commented on HDFS-9027:
--
SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2321 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2321/])
HDFS-9027. Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method.
(Contributed by Mingliang Liu) (arp: rev
15a557fcfec5eceedde9f1597385d5d3b01b2cd7)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
*
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java
> Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
> ---
>
> Key: HDFS-9027
> URL: https://issues.apache.org/jira/browse/HDFS-9027
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HDFS-9027.000.patch, HDFS-9027.001.patch
>
>
> In method {{isLazyPersist()}}, the {{org.apache.hadoop.hdfs.DataStreamer}}
> class checks whether the HDFS file is lazy persist. It does two things:
> 1. Create a class-wide _static_ {{BlockStoragePolicySuite}} object, which
> builds an array of {{BlockStoragePolicy}} internally
> 2. Get a block storage policy object from the {{blockStoragePolicySuite}} by
> policy name {{HdfsConstants.MEMORY_STORAGE_POLICY_NAME}}
> This has two side effects:
> 1. Takes time to iterate the pre-built block storage policy array in order to
> find the _same_ policy every time whose id matters only (as we need to
> compare the file status policy id with lazy persist policy id)
> 2. {{DataStreamer}} class imports {{BlockStoragePolicySuite}}. The former
> should be moved to {{hadoop-hdfs-client}} module, while the latter can stay
> in {{hadoop-hdfs}} module.
> Actually, we have the block storage policy IDs, which can be used to compare
> with HDFS file status' policy id, as following:
> {code}
> static boolean isLazyPersist(HdfsFileStatus stat) {
> return stat.getStoragePolicy() == HdfsConstants.MEMORY_STORAGE_POLICY_ID;
> }
> {code}
> This way, we only need to move the block storage policies' IDs from
> {{HdfsServerConstant}} ({{hadoop-hdfs}} module) to {{HdfsConstants}}
> ({{hadoop-hdfs-client}} module).
> Another reason we should move those block storage policy IDs is that the
> block storage policy names were moved to {{HdfsConstants}} already.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9027) Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
[
https://issues.apache.org/jira/browse/HDFS-9027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741571#comment-14741571
]
Hudson commented on HDFS-9027:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #359 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/359/])
HDFS-9027. Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method.
(Contributed by Mingliang Liu) (arp: rev
15a557fcfec5eceedde9f1597385d5d3b01b2cd7)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
*
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
> Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
> ---
>
> Key: HDFS-9027
> URL: https://issues.apache.org/jira/browse/HDFS-9027
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HDFS-9027.000.patch, HDFS-9027.001.patch
>
>
> In method {{isLazyPersist()}}, the {{org.apache.hadoop.hdfs.DataStreamer}}
> class checks whether the HDFS file is lazy persist. It does two things:
> 1. Create a class-wide _static_ {{BlockStoragePolicySuite}} object, which
> builds an array of {{BlockStoragePolicy}} internally
> 2. Get a block storage policy object from the {{blockStoragePolicySuite}} by
> policy name {{HdfsConstants.MEMORY_STORAGE_POLICY_NAME}}
> This has two side effects:
> 1. Takes time to iterate the pre-built block storage policy array in order to
> find the _same_ policy every time whose id matters only (as we need to
> compare the file status policy id with lazy persist policy id)
> 2. {{DataStreamer}} class imports {{BlockStoragePolicySuite}}. The former
> should be moved to {{hadoop-hdfs-client}} module, while the latter can stay
> in {{hadoop-hdfs}} module.
> Actually, we have the block storage policy IDs, which can be used to compare
> with HDFS file status' policy id, as following:
> {code}
> static boolean isLazyPersist(HdfsFileStatus stat) {
> return stat.getStoragePolicy() == HdfsConstants.MEMORY_STORAGE_POLICY_ID;
> }
> {code}
> This way, we only need to move the block storage policies' IDs from
> {{HdfsServerConstant}} ({{hadoop-hdfs}} module) to {{HdfsConstants}}
> ({{hadoop-hdfs-client}} module).
> Another reason we should move those block storage policy IDs is that the
> block storage policy names were moved to {{HdfsConstants}} already.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9027) Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
[
https://issues.apache.org/jira/browse/HDFS-9027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741242#comment-14741242
]
Hudson commented on HDFS-9027:
--
FAILURE: Integrated in Hadoop-trunk-Commit #8435 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/8435/])
HDFS-9027. Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method.
(Contributed by Mingliang Liu) (arp: rev
15a557fcfec5eceedde9f1597385d5d3b01b2cd7)
*
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HdfsServerConstants.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
> Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
> ---
>
> Key: HDFS-9027
> URL: https://issues.apache.org/jira/browse/HDFS-9027
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HDFS-9027.000.patch, HDFS-9027.001.patch
>
>
> In method {{isLazyPersist()}}, the {{org.apache.hadoop.hdfs.DataStreamer}}
> class checks whether the HDFS file is lazy persist. It does two things:
> 1. Create a class-wide _static_ {{BlockStoragePolicySuite}} object, which
> builds an array of {{BlockStoragePolicy}} internally
> 2. Get a block storage policy object from the {{blockStoragePolicySuite}} by
> policy name {{HdfsConstants.MEMORY_STORAGE_POLICY_NAME}}
> This has two side effects:
> 1. Takes time to iterate the pre-built block storage policy array in order to
> find the _same_ policy every time whose id matters only (as we need to
> compare the file status policy id with lazy persist policy id)
> 2. {{DataStreamer}} class imports {{BlockStoragePolicySuite}}. The former
> should be moved to {{hadoop-hdfs-client}} module, while the latter can stay
> in {{hadoop-hdfs}} module.
> Actually, we have the block storage policy IDs, which can be used to compare
> with HDFS file status' policy id, as following:
> {code}
> static boolean isLazyPersist(HdfsFileStatus stat) {
> return stat.getStoragePolicy() == HdfsConstants.MEMORY_STORAGE_POLICY_ID;
> }
> {code}
> This way, we only need to move the block storage policies' IDs from
> {{HdfsServerConstant}} ({{hadoop-hdfs}} module) to {{HdfsConstants}}
> ({{hadoop-hdfs-client}} module).
> Another reason we should move those block storage policy IDs is that the
> block storage policy names were moved to {{HdfsConstants}} already.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9027) Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
[
https://issues.apache.org/jira/browse/HDFS-9027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739643#comment-14739643
]
Arpit Agarwal commented on HDFS-9027:
-
Hi [~liuml07], thanks for this improvement.
# Let's remove the {{HdfsServerConstants#*_STORAGE_POLICY_ID}} constants as
they are unused.
# The Preconditions.checkNotNull is not required in
{{DataStreamer#isLazyPersist}}. If stat is null the next line will throw NPE
with the same effect.
Other than that, the patch looks good.
> Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
> ---
>
> Key: HDFS-9027
> URL: https://issues.apache.org/jira/browse/HDFS-9027
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-9027.000.patch
>
>
> In method {{isLazyPersist()}}, the {{org.apache.hadoop.hdfs.DataStreamer}}
> class checks whether the HDFS file is lazy persist. It does two things:
> 1. Create a class-wide _static_ {{BlockStoragePolicySuite}} object, which
> builds an array of {{BlockStoragePolicy}} internally
> 2. Get a block storage policy object from the {{blockStoragePolicySuite}} by
> policy name {{HdfsConstants.MEMORY_STORAGE_POLICY_NAME}}
> This has two side effects:
> 1. Takes time to iterate the pre-built block storage policy array in order to
> find the _same_ policy every time whose id matters only (as we need to
> compare the file status policy id with lazy persist policy id)
> 2. {{DataStreamer}} class imports {{BlockStoragePolicySuite}}. The former
> should be moved to {{hadoop-hdfs-client}} module, while the latter can stay
> in {{hadoop-hdfs}} module.
> Actually, we have the block storage policy IDs, which can be used to compare
> with HDFS file status' policy id, as following:
> {code}
> static boolean isLazyPersist(HdfsFileStatus stat) {
> return stat.getStoragePolicy() == HdfsConstants.MEMORY_STORAGE_POLICY_ID;
> }
> {code}
> This way, we only need to move the block storage policies' IDs from
> {{HdfsServerConstant}} ({{hadoop-hdfs}} module) to {{HdfsConstants}}
> ({{hadoop-hdfs-client}} module).
> Another reason we should move those block storage policy IDs is that the
> block storage policy names were moved to {{HdfsConstants}} already.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9027) Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
[
https://issues.apache.org/jira/browse/HDFS-9027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739765#comment-14739765
]
Arpit Agarwal commented on HDFS-9027:
-
+1 for the v1 patch pending Jenkins. Thanks for addressing the comments.
> Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
> ---
>
> Key: HDFS-9027
> URL: https://issues.apache.org/jira/browse/HDFS-9027
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-9027.000.patch, HDFS-9027.001.patch
>
>
> In method {{isLazyPersist()}}, the {{org.apache.hadoop.hdfs.DataStreamer}}
> class checks whether the HDFS file is lazy persist. It does two things:
> 1. Create a class-wide _static_ {{BlockStoragePolicySuite}} object, which
> builds an array of {{BlockStoragePolicy}} internally
> 2. Get a block storage policy object from the {{blockStoragePolicySuite}} by
> policy name {{HdfsConstants.MEMORY_STORAGE_POLICY_NAME}}
> This has two side effects:
> 1. Takes time to iterate the pre-built block storage policy array in order to
> find the _same_ policy every time whose id matters only (as we need to
> compare the file status policy id with lazy persist policy id)
> 2. {{DataStreamer}} class imports {{BlockStoragePolicySuite}}. The former
> should be moved to {{hadoop-hdfs-client}} module, while the latter can stay
> in {{hadoop-hdfs}} module.
> Actually, we have the block storage policy IDs, which can be used to compare
> with HDFS file status' policy id, as following:
> {code}
> static boolean isLazyPersist(HdfsFileStatus stat) {
> return stat.getStoragePolicy() == HdfsConstants.MEMORY_STORAGE_POLICY_ID;
> }
> {code}
> This way, we only need to move the block storage policies' IDs from
> {{HdfsServerConstant}} ({{hadoop-hdfs}} module) to {{HdfsConstants}}
> ({{hadoop-hdfs-client}} module).
> Another reason we should move those block storage policy IDs is that the
> block storage policy names were moved to {{HdfsConstants}} already.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9027) Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
[
https://issues.apache.org/jira/browse/HDFS-9027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14740219#comment-14740219
]
Hadoop QA commented on HDFS-9027:
-
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 19m 53s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 54s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 8s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 2m 35s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 26s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 13s | Pre-build of native portion |
| {color:green}+1{color} | hdfs tests | 163m 17s | Tests passed in hadoop-hdfs.
|
| {color:green}+1{color} | hdfs tests | 0m 29s | Tests passed in
hadoop-hdfs-client. |
| | | 214m 33s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12755255/HDFS-9027.001.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f103a70 |
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12384/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| hadoop-hdfs-client test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12384/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12384/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12384/console |
This message was automatically generated.
> Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
> ---
>
> Key: HDFS-9027
> URL: https://issues.apache.org/jira/browse/HDFS-9027
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-9027.000.patch, HDFS-9027.001.patch
>
>
> In method {{isLazyPersist()}}, the {{org.apache.hadoop.hdfs.DataStreamer}}
> class checks whether the HDFS file is lazy persist. It does two things:
> 1. Create a class-wide _static_ {{BlockStoragePolicySuite}} object, which
> builds an array of {{BlockStoragePolicy}} internally
> 2. Get a block storage policy object from the {{blockStoragePolicySuite}} by
> policy name {{HdfsConstants.MEMORY_STORAGE_POLICY_NAME}}
> This has two side effects:
> 1. Takes time to iterate the pre-built block storage policy array in order to
> find the _same_ policy every time whose id matters only (as we need to
> compare the file status policy id with lazy persist policy id)
> 2. {{DataStreamer}} class imports {{BlockStoragePolicySuite}}. The former
> should be moved to {{hadoop-hdfs-client}} module, while the latter can stay
> in {{hadoop-hdfs}} module.
> Actually, we have the block storage policy IDs, which can be used to compare
> with HDFS file status' policy id, as following:
> {code}
> static boolean isLazyPersist(HdfsFileStatus stat) {
> return stat.getStoragePolicy() == HdfsConstants.MEMORY_STORAGE_POLICY_ID;
> }
> {code}
> This way, we only need to move the block storage policies' IDs from
> {{HdfsServerConstant}} ({{hadoop-hdfs}} module) to {{HdfsConstants}}
> ({{hadoop-hdfs-client}} module).
> Another reason we should move those block storage policy IDs is that the
> block storage policy names were moved to {{HdfsConstants}} already.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9027) Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
[
https://issues.apache.org/jira/browse/HDFS-9027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737432#comment-14737432
]
Hadoop QA commented on HDFS-9027:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 19m 25s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 56s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 3s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 2m 32s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 22s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 10s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 164m 21s | Tests failed in hadoop-hdfs. |
| {color:green}+1{color} | hdfs tests | 0m 29s | Tests passed in
hadoop-hdfs-client. |
| | | 214m 52s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.TestDFSPermission |
| | hadoop.hdfs.web.TestWebHDFSOAuth2 |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12754265/HDFS-9027.000.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 0113e45 |
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12356/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| hadoop-hdfs-client test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12356/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12356/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12356/console |
This message was automatically generated.
> Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method
> ---
>
> Key: HDFS-9027
> URL: https://issues.apache.org/jira/browse/HDFS-9027
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-9027.000.patch
>
>
> In method {{isLazyPersist()}}, the {{org.apache.hadoop.hdfs.DataStreamer}}
> class checks whether the HDFS file is lazy persist. It does two things:
> 1. Create a class-wide _static_ {{BlockStoragePolicySuite}} object, which
> builds an array of {{BlockStoragePolicy}} internally
> 2. Get a block storage policy object from the {{blockStoragePolicySuite}} by
> policy name {{HdfsConstants.MEMORY_STORAGE_POLICY_NAME}}
> This has two side effects:
> 1. Takes time to iterate the pre-built block storage policy array in order to
> find the _same_ policy every time whose id matters only (as we need to
> compare the file status policy id with lazy persist policy id)
> 2. {{DataStreamer}} class imports {{BlockStoragePolicySuite}}. The former
> should be moved to {{hadoop-hdfs-client}} module, while the latter can stay
> in {{hadoop-hdfs}} module.
> Actually, we have the block storage policy IDs, which can be used to compare
> with HDFS file status' policy id, as following:
> {code}
> static boolean isLazyPersist(HdfsFileStatus stat) {
> return stat.getStoragePolicy() == HdfsConstants.MEMORY_STORAGE_POLICY_ID;
> }
> {code}
> This way, we only need to move the block storage policies' IDs from
> {{HdfsServerConstant}} ({{hadoop-hdfs}} module) to {{HdfsConstants}}
> ({{hadoop-hdfs-client}} module).
> Another reason we should move those block storage policy IDs is that the
> block storage policy names were moved to {{HdfsConstants}} already.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9027) Refactor o.a.h.hdfs.DataStreamer$isLazyPersist() method
[
https://issues.apache.org/jira/browse/HDFS-9027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731500#comment-14731500
]
Hadoop QA commented on HDFS-9027:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 17m 53s | Findbugs (version 3.0.0)
appears to be broken on trunk. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 1s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 52s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 3s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 2m 14s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 35s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 25s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 16s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 0m 22s | Tests failed in hadoop-hdfs. |
| {color:green}+1{color} | hdfs tests | 0m 27s | Tests passed in
hadoop-hdfs-client. |
| | | 49m 11s | |
\\
\\
|| Reason || Tests ||
| Failed build | hadoop-hdfs |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12754265/HDFS-9027.000.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / e1feaf6 |
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12317/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| hadoop-hdfs-client test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12317/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12317/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12317/console |
This message was automatically generated.
> Refactor o.a.h.hdfs.DataStreamer$isLazyPersist() method
> ---
>
> Key: HDFS-9027
> URL: https://issues.apache.org/jira/browse/HDFS-9027
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-9027.000.patch
>
>
> In method {{isLazyPersist()}}, the {{org.apache.hadoop.hdfs.DataStreamer}}
> class checks whether the HDFS file is lazy persist. It does two things:
> 1. Create a class-wide _static_ {{BlockStoragePolicySuite}} object, which
> builds an array of {{BlockStoragePolicy}} internally
> 2. Get a block storage policy object from the {{blockStoragePolicySuite}} by
> policy name {{HdfsConstants.MEMORY_STORAGE_POLICY_NAME}}
> This has two side effects:
> 1. Takes time to iterate the pre-built block storage policy array in order to
> find the _same_ policy every time whose id matters only (as we need to
> compare the file status policy id with lazy persist policy id)
> 2. {{DataStreamer}} class imports {{BlockStoragePolicySuite}}. The former
> should be moved to {{hadoop-hdfs-client}} module, while the latter can stay
> in {{hadoop-hdfs}} module.
> Actually, we have the block storage policy IDs, which can be used to compare
> with HDFS file status' policy id, as following:
> {code}
> static boolean isLazyPersist(HdfsFileStatus stat) {
> return stat.getStoragePolicy() == HdfsConstants.MEMORY_STORAGE_POLICY_ID;
> }
> {code}
> This way, we only need to move the block storage policies' IDs from
> {{HdfsServerConstant}} ({{hadoop-hdfs}} module) to {{HdfsConstants}}
> ({{hadoop-hdfs-client}} module).
> Another reason we should move those block storage policy IDs is that the
> block storage policy names were moved to {{HdfsConstants}} already.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
