Hi all, I see in o.a.h.hdfs.server.blockmanagement.SequentialBlockIdGenerator that the low 30 bits of the Block ID are reserved. This was set out in HDFS-4645 [1,2]:
“”” We do not change the block ID of any existing blocks on upgrade. Such existing blocks whose IDs were randomly generated are subsequently referred to as legacy blocks. Henceforth block IDs will be allocated sequentially starting from a fixed constant e.g. 2^30. “”” This doesn’t really follow since a uniform distribution wouldn’t have made block IDs all that likely to have populated those low 30 bits. My only guess is that the pseudorandom number generator in the legacy block ID generation was not uniform across the 64 bit block ID space. In the Jira, Suresh suggested only 16 bits: “”” I think we could reserve few block IDs say 0-65535 and start generating from 65535. When it reaches some max, we could rollover to negative numbers. That is a decision that can be made in the future. “”” So I’m curious why the initial range was skipped if the pseudorandom number block ID generator wouldn’t really have favoured the low range of block IDs. And why was the initial range of 16 bits changed to skip the initial 30 bits? Thanks for the help in understanding this! Yours, Ewan [1] https://issues.apache.org/jira/browse/HDFS-4645 [2] https://issues.apache.org/jira/secure/attachment/12589172/SequentialblockIDallocation.pdf Western Digital Corporation (and its subsidiaries) E-mail Confidentiality Notice & Disclaimer: This e-mail and any files transmitted with it may contain confidential or legally privileged information of WDC and/or its affiliates, and are intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited. If you have received this e-mail in error, please notify the sender immediately and delete the e-mail in its entirety from your system.
