[jira] [Commented] (HDFS-13270) RBF: Router audit logger
[ https://issues.apache.org/jira/browse/HDFS-13270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17865892#comment-17865892 ] TIsNotT commented on HDFS-13270: This patch is very good for us, but there was a mistake where "lbs=" was missing. !image-2024-07-15-14-46-54-168.png|width=636,height=102! > RBF: Router audit logger > > > Key: HDFS-13270 > URL: https://issues.apache.org/jira/browse/HDFS-13270 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Affects Versions: 3.2.0 >Reporter: Baolong Mao >Assignee: Hemanth Boyina >Priority: Major > Attachments: HDFS-13270.001.patch, HDFS-13270.002.patch, > HDFS-13270.003.patch, image-2024-07-15-14-46-54-168.png > > > We can use router auditlogger to log the client info and cmd, because the > FSNamesystem#Auditlogger's log think the client are all from router. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13270) RBF: Router audit logger
[ https://issues.apache.org/jira/browse/HDFS-13270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TIsNotT updated HDFS-13270: --- Attachment: image-2024-07-15-14-46-54-168.png > RBF: Router audit logger > > > Key: HDFS-13270 > URL: https://issues.apache.org/jira/browse/HDFS-13270 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Affects Versions: 3.2.0 >Reporter: Baolong Mao >Assignee: Hemanth Boyina >Priority: Major > Attachments: HDFS-13270.001.patch, HDFS-13270.002.patch, > HDFS-13270.003.patch, image-2024-07-15-14-46-54-168.png > > > We can use router auditlogger to log the client info and cmd, because the > FSNamesystem#Auditlogger's log think the client are all from router. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-17162) RBF: Add missing comments in StateStoreService
[ https://issues.apache.org/jira/browse/HDFS-17162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TIsNotT updated HDFS-17162: --- Description: StateStoreService lacks an introduction to StateStoreFileSystemImpl and StateStoreMySQLImpl [link HDFS-16943|https://issues.apache.org/jira/projects/HDFS/issues/HDFS-16943?filter=allissues]. (was: StateStoreService lacks an introduction to StateStoreFileSystemImpl and StateStoreMySQLImpl.) > RBF: Add missing comments in StateStoreService > -- > > Key: HDFS-17162 > URL: https://issues.apache.org/jira/browse/HDFS-17162 > Project: Hadoop HDFS > Issue Type: Improvement > Components: rbf >Reporter: TIsNotT >Priority: Minor > > StateStoreService lacks an introduction to StateStoreFileSystemImpl and > StateStoreMySQLImpl [link > HDFS-16943|https://issues.apache.org/jira/projects/HDFS/issues/HDFS-16943?filter=allissues]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-17162) RBF: Add missing comments in StateStoreService
TIsNotT created HDFS-17162: -- Summary: RBF: Add missing comments in StateStoreService Key: HDFS-17162 URL: https://issues.apache.org/jira/browse/HDFS-17162 Project: Hadoop HDFS Issue Type: Improvement Components: rbf Reporter: TIsNotT StateStoreService lacks an introduction to StateStoreFileSystemImpl and StateStoreMySQLImpl. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-17155) Fixed small doc in mounttable class
[ https://issues.apache.org/jira/browse/HDFS-17155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TIsNotT resolved HDFS-17155. Release Note: I mistook the branch Resolution: Won't Fix > Fixed small doc in mounttable class > --- > > Key: HDFS-17155 > URL: https://issues.apache.org/jira/browse/HDFS-17155 > Project: Hadoop HDFS > Issue Type: Bug > Components: rbf >Reporter: TIsNotT >Priority: Minor > > /** >* Set the destination paths. >* >* @param paths Destination paths. >*/ > public abstract void setDestinations(List dests); > The annotation value is wrong , it shuld be dests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-17155) Fixed small doc in mounttable class
TIsNotT created HDFS-17155: -- Summary: Fixed small doc in mounttable class Key: HDFS-17155 URL: https://issues.apache.org/jira/browse/HDFS-17155 Project: Hadoop HDFS Issue Type: Bug Components: rbf Reporter: TIsNotT /** * Set the destination paths. * * @param paths Destination paths. */ public abstract void setDestinations(List dests); The annotation value is wrong , it shuld be dests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12643) HDFS maintenance state behaviour is confusing and not well documented
[
https://issues.apache.org/jira/browse/HDFS-12643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421255#comment-17421255
]
TisNotT edited comment on HDFS-12643 at 10/11/21, 3:46 AM:
---
it is confuse to me too.So I am finding whether there is a way to set special
datanodes to maintenance state or not. If not, maybe I will dev one for my
company.
was (Author: sharpshow):
it is confuse to me too.So I am finding whether there is an api to set special
datanodes to maintenance state or not. If not,maybe I will dev one for my
company.
> HDFS maintenance state behaviour is confusing and not well documented
> -
>
> Key: HDFS-12643
> URL: https://issues.apache.org/jira/browse/HDFS-12643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation, namenode
>Reporter: Andre Araujo
>Priority: Major
>
> The current implementation of the HDFS maintenance state feature is confusing
> and error-prone. The documentation is missing important information that's
> required for the correct use of the feature.
> For example, if the Hadoop admin wants to put a single node in maintenance
> state, he/she can add a single entry to the maintenance file with the
> contents:
> {code}
> {
>"hostName": "host-1.example.com",
>"adminState": "IN_MAINTENANCE",
>"maintenanceExpireTimeInMS": 1507663698000
> }
> {code}
> Let's say now that the actual maintenance finished well before the set
> expiration time and the Hadoop admin wants to bring the node back to NORMAL
> state. It would be natural to simply change the state of the node, as show
> below, and run another refresh:
> {code}
> {
>"hostName": "host-1.example.com",
>"adminState": "NORMAL"
> }
> {code}
> The configuration file above, though, not only take the node {{host-1}} out
> of maintenance state but it also *blacklists all the other DataNodes*. This
> behaviour seems inconsistent to me and is due to {{emptyInServiceNodeLists}}
> being set to {{false}}
> [here|https://github.com/apache/hadoop/blob/230b85d5865b7e08fb7aaeab45295b5b966011ef/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CombinedHostFileManager.java#L80]
> only when there is at least one node with {{adminState = NORMAL}} listed in
> the file.
> I believe that it would be more consistent, and less error prone, to simply
> implement the following:
> * If the dfs.hosts file is empty, all nodes are allowed and in normal state
> * If the file is not empty, any host *not* listed in the file is
> *blacklisted*, regardless of the state of the hosts listed in the file.
> Regardless of the implementation being changed or not, the documentation also
> needs to be updated to ensure the readers know of the caveats mentioned above.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12643) HDFS maintenance state behaviour is confusing and not well documented
[
https://issues.apache.org/jira/browse/HDFS-12643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421255#comment-17421255
]
TisNotT commented on HDFS-12643:
it is confuse to me too.So I am finding whether there is an api to set special
datanodes to maintenance state or not. If not,maybe I will dev one for my
company.
> HDFS maintenance state behaviour is confusing and not well documented
> -
>
> Key: HDFS-12643
> URL: https://issues.apache.org/jira/browse/HDFS-12643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation, namenode
>Reporter: Andre Araujo
>Priority: Major
>
> The current implementation of the HDFS maintenance state feature is confusing
> and error-prone. The documentation is missing important information that's
> required for the correct use of the feature.
> For example, if the Hadoop admin wants to put a single node in maintenance
> state, he/she can add a single entry to the maintenance file with the
> contents:
> {code}
> {
>"hostName": "host-1.example.com",
>"adminState": "IN_MAINTENANCE",
>"maintenanceExpireTimeInMS": 1507663698000
> }
> {code}
> Let's say now that the actual maintenance finished well before the set
> expiration time and the Hadoop admin wants to bring the node back to NORMAL
> state. It would be natural to simply change the state of the node, as show
> below, and run another refresh:
> {code}
> {
>"hostName": "host-1.example.com",
>"adminState": "NORMAL"
> }
> {code}
> The configuration file above, though, not only take the node {{host-1}} out
> of maintenance state but it also *blacklists all the other DataNodes*. This
> behaviour seems inconsistent to me and is due to {{emptyInServiceNodeLists}}
> being set to {{false}}
> [here|https://github.com/apache/hadoop/blob/230b85d5865b7e08fb7aaeab45295b5b966011ef/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CombinedHostFileManager.java#L80]
> only when there is at least one node with {{adminState = NORMAL}} listed in
> the file.
> I believe that it would be more consistent, and less error prone, to simply
> implement the following:
> * If the dfs.hosts file is empty, all nodes are allowed and in normal state
> * If the file is not empty, any host *not* listed in the file is
> *blacklisted*, regardless of the state of the hosts listed in the file.
> Regardless of the implementation being changed or not, the documentation also
> needs to be updated to ensure the readers know of the caveats mentioned above.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
