[jira] [Created] (HDFS-17522) JournalNode web interfaces lack configs for X-FRAME-OPTIONS protection

2024-05-12 Thread wangzhihui (Jira) 
wangzhihui created HDFS-17522:
-

 Summary: JournalNode web interfaces lack configs for 
X-FRAME-OPTIONS protection
 Key: HDFS-17522
 URL: https://issues.apache.org/jira/browse/HDFS-17522
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: journal-node
Affects Versions: 3.0.0-alpha1
Reporter: wangzhihui






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17486) VIO: dumpXattrs logic optimization

2024-04-22 Thread wangzhihui (Jira) 
wangzhihui created HDFS-17486:
-

 Summary: VIO: dumpXattrs logic optimization
 Key: HDFS-17486
 URL: https://issues.apache.org/jira/browse/HDFS-17486
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: hdfs
Affects Versions: 3.3.3, 3.2.0
Reporter: wangzhihui


The dumpXattrs logic in VIO should use FSImageFormatPBINode.Loader.loadXAttrs() 
to get the Xattrs attribute for easy maintenance.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17463) Title ： Support the switch StringTable Split ID feature

2024-04-12 Thread wangzhihui (Jira) 
wangzhihui created HDFS-17463:
-

 Summary: Title ： Support the switch StringTable Split ID feature
 Key: HDFS-17463
 URL: https://issues.apache.org/jira/browse/HDFS-17463
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: namenode
Affects Versions: 3.3.4, 3.3.3, 3.3.5, 3.2.0
Reporter: wangzhihui
 Attachments: Image_struct.png, error.png

desc:
 * 
Hadoop 3.2 introduced optimization features for HDFS StringTable 
(b60ca37914b22550e3630fa02742d40697decb3), It resulted in lower versions of 
Hadoop upgraded to 3.2 and later versions not supporting downgrade operations. 

!error.png!
 * This issue has also been discussed in HDFS-14831, and it is recommended to 
revert the feature, but it cannot fundamentally solve the problem。
 * 
Therefore, we have added an optimization to support downgrading

 

Solution:
 * First, we will add the "dfs. image. save. splitId. stringTable" conf switch 
"StringTable optimization feature" is enabled

 * When the conf value is false, an Image file compatible with lower versions 
of HDFS is generated to support downgrading.
 * 
The difference in HDFS Image file format between Hadoop 3.1.1 and Hadoop 3.2 is 
shown in the following figure.
 * With the sub-sections feature introduced in HDFS-14617, Protobuf can support 
compatible reading.
 * 
The data structure causing incompatible differences is mainly StringTable.

!Image_struct.png|width=396,height=163!
 * In "dfs.image.save.splitId.stringTable = false " the Id generation order of 
StringTable starts from 0 to Integer.Max. When true, the Id value range follows 
the latest rules.
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17356) RBF Add Configuration dfs.federation.router.ns.name Optimization

2024-01-25 Thread wangzhihui (Jira) 
wangzhihui created HDFS-17356:
-

 Summary: RBF Add Configuration dfs.federation.router.ns.name 
Optimization
 Key: HDFS-17356
 URL: https://issues.apache.org/jira/browse/HDFS-17356
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: dfs, rbf
Reporter: wangzhihui
 Fix For: 3.3.0


    When enabling RBF federation in HDFS, when the HDFS server and RBFClient 
share the same configuration and the HDFS server (NameNode、ZKFC) and RBFClient 
are on the same node, the following exception occurs, causing NameNode to fail 
to start; The reason is that the NS of the Router service has been added to the 
dfs.nameservices list. When NameNode starts, it obtains the NS that the current 
node belongs to. However, it is found that there are multiple NS that cannot be 
recognized and cannot pass the verification of existing logic, ultimately 
resulting in NameNode startup failure. Currently, we can only solve this 
problem by isolating the hdfs-site.xml of RouterClient and NameNode. However, 
grouping configuration is not conducive to our unified management of cluster 
configuration. Therefore, we propose a new solution to solve this problem 
better.
{code:java}
// code placeholder
2023-10-30 15:53:24,613 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: 
registered UNIX signal handlers for [TERM, HUP, INT]
2023-10-30 15:53:24,672 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: 
createNameNode []
2023-10-30 15:53:24,760 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: 
Loaded properties from hadoop-metrics2.properties
2023-10-30 15:53:24,842 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: 
Scheduled Metric snapshot period at 10 second(s).
2023-10-30 15:53:24,842 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: 
NameNode metrics system started
2023-10-30 15:53:24,868 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: 
Failed to start namenode.
org.apache.hadoop.HadoopIllegalArgumentException: Configuration has multiple 
addresses that match local node's address. Please configure the system with 
dfs.nameservice.id and dfs.ha.namenode.id
        at org.apache.hadoop.hdfs.DFSUtil.getSuffixIDs(DFSUtil.java:1257)
        at org.apache.hadoop.hdfs.DFSUtil.getNameServiceId(DFSUtil.java:1158)
        at 
org.apache.hadoop.hdfs.DFSUtil.getNamenodeNameServiceId(DFSUtil.java:1113)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.getNameServiceId(NameNode.java:1822)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:1005)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:995)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1769)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1834)
2023-10-30 15:53:24,870 INFO org.apache.hadoop.util.ExitUtil: Exiting with 
status 1: org.apache.hadoop.HadoopIllegalArgumentException: Configuration has 
multiple addresses that match local node's address. Please configure the system 
with dfs.nameservice.id and dfs.ha.name
node.id
2023-10-30 15:53:24,874 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: 
SHUTDOWN_MSG: {code}
Solution 

Add dfs.federation.router.ns.name configuration in hdfs-site.xml to mark the 
Router NS name. and filter out Router NS during NameNode or ZKFC startup to 
avoid this issue.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org