[jira] [Reopened] (HDFS-17575) SaslDataTransferClient should use SaslParticipant to create messages

2024-07-21 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze reopened HDFS-17575: --- The [pull request 6933 |https://github.com/apache/hadoop/pull/6933] has caused test failure. Reverted

[jira] [Resolved] (HDFS-17576) Support user defined auth Callback

2024-07-20 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-17576. --- Fix Version/s: 3.3.7 Hadoop Flags: Reviewed Resolution: Fixed The pull request is now

[jira] [Resolved] (HDFS-17575) SaslDataTransferClient should use SaslParticipant to create messages

2024-07-20 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-17575. --- Fix Version/s: 3.3.7 Resolution: Fixed The pull request is now merged

[jira] [Created] (HDFS-17575) SaslDataTransferClient should use SaslParticipant to create messages

2024-07-09 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-17575: - Summary: SaslDataTransferClient should use SaslParticipant to create messages Key: HDFS-17575 URL: https://issues.apache.org/jira/browse/HDFS-17575 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-10535) Rename AsyncDistributedFileSystem

2024-07-09 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-10535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-10535. --- Resolution: Won't Fix This JIRA became stale. > Rename AsyncDistributedFileSys

[jira] [Resolved] (HDFS-11948) Ozone: change TestRatisManager to check cluster with data

2024-07-09 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-11948. --- Resolution: Won't Fix This JIRA became stale. > Ozone: change TestRatisManager to check clus

[jira] [Resolved] (HDFS-11734) Ozone: provide a way to validate ContainerCommandRequestProto

2024-07-09 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-11734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-11734. --- Resolution: Won't Fix This JIRA became stale. > Ozone: provide a way to valid

[jira] [Resolved] (HDFS-11735) Ozone: In Ratis, leader should validate ContainerCommandRequestProto before propagating it to followers

2024-07-09 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-11735. --- Resolution: Won't Fix This JIRA became stale. > Ozone: In Ratis, leader should valid

[jira] [Resolved] (HDFS-17528) FsImageValidation: set txid when saving a new image

2024-06-18 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-17528. --- Fix Version/s: 3.5.0 Hadoop Flags: Reviewed Resolution: Fixed Th pull request is now

[jira] [Created] (HDFS-17549) SecretManager should not hardcode HMAC algorithm

2024-06-07 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-17549: - Summary: SecretManager should not hardcode HMAC algorithm Key: HDFS-17549 URL: https://issues.apache.org/jira/browse/HDFS-17549 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-17528) FsImageValidation: set txid when saving a new image

2024-05-14 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-17528: - Summary: FsImageValidation: set txid when saving a new image Key: HDFS-17528 URL: https://issues.apache.org/jira/browse/HDFS-17528 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-17495) Change FSNamesystem.digest to use a configurable algorithm.

2024-04-22 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-17495: - Summary: Change FSNamesystem.digest to use a configurable algorithm. Key: HDFS-17495 URL: https://issues.apache.org/jira/browse/HDFS-17495 Project: Hadoop HDFS

Re: HDFS test builds always fails

2024-03-13 Thread Tsz Wo Sze
Thanks a lot for fixing this!  I saw "random" failures multiple times in  https://github.com/apache/hadoop/pull/6549 It can pass all the tests now. Tsz-Wo On Tuesday, March 12, 2024 at 01:03:14 PM PDT, Ayush Saxena wrote: We are sorted now: the daily build from yesterday [1] shows 0

[jira] [Resolved] (HDFS-17403) jenkins build failing for HDFS 3.5.0-SNAPSHOT

2024-03-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-17403. --- Resolution: Not A Problem Resolving ... > jenkins build failing for HDFS 3.5.0-SNAPS

[jira] [Created] (HDFS-17380) FsImageValidation: remove inaccessible nodes

2024-02-13 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-17380: - Summary: FsImageValidation: remove inaccessible nodes Key: HDFS-17380 URL: https://issues.apache.org/jira/browse/HDFS-17380 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-17164) During rolling upgrade, datanode should copy-on-append when bumpReplicaGS

2023-08-18 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-17164: - Summary: During rolling upgrade, datanode should copy-on-append when bumpReplicaGS Key: HDFS-17164 URL: https://issues.apache.org/jira/browse/HDFS-17164 Project: Hadoop

[jira] [Resolved] (HDFS-17045) File renamed from a snapshottable dir to a non-snapshottable dir cannot be deleted.

2023-06-14 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-17045. --- Fix Version/s: 3.4.0 Resolution: Fixed The pull request is now merged. > File renamed f

[jira] [Created] (HDFS-17045) File renamed from a snapshottable dir to a non-snapshottable dir cannot be deleted.

2023-06-12 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-17045: - Summary: File renamed from a snapshottable dir to a non-snapshottable dir cannot be deleted. Key: HDFS-17045 URL: https://issues.apache.org/jira/browse/HDFS-17045 Project

[jira] [Resolved] (HDFS-17010) Add a subtree test to TestSnapshotDiffReport

2023-05-18 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-17010. --- Fix Version/s: 3.4.0 Resolution: Fixed The pull request is now merged. > Add a subtree t

[jira] [Created] (HDFS-17010) Add a subtree test to TestSnapshotDiffReport

2023-05-14 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-17010: - Summary: Add a subtree test to TestSnapshotDiffReport Key: HDFS-17010 URL: https://issues.apache.org/jira/browse/HDFS-17010 Project: Hadoop HDFS Issue Type: Test

[jira] [Resolved] (HDFS-16972) Delete a snapshot may deleteCurrentFile

2023-04-27 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-16972. --- Fix Version/s: 3.4.0 Resolution: Fixed The pull request is now merged. > Delete a snaps

[jira] [Resolved] (HDFS-16975) FileWithSnapshotFeature.isCurrentFileDeleted is not reloaded from FSImage.

2023-04-24 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-16975. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed The pull request is now

[jira] [Created] (HDFS-16975) FileWithSnapshotFeature.isCurrentFileDeleted is not serialized to FSImage.

2023-04-11 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-16975: - Summary: FileWithSnapshotFeature.isCurrentFileDeleted is not serialized to FSImage. Key: HDFS-16975 URL: https://issues.apache.org/jira/browse/HDFS-16975 Project: Hadoop

[jira] [Created] (HDFS-16972) Delete a snapshot may deleteCurrentFile

2023-04-04 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-16972: - Summary: Delete a snapshot may deleteCurrentFile Key: HDFS-16972 URL: https://issues.apache.org/jira/browse/HDFS-16972 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-16963) Remove the unnecessary use of Optional from DistributedFileSystem

2023-03-23 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-16963: - Summary: Remove the unnecessary use of Optional from DistributedFileSystem Key: HDFS-16963 URL: https://issues.apache.org/jira/browse/HDFS-16963 Project: Hadoop HDFS

Re: [DISCUSS] Move HDFS specific APIs to FileSystem abstration

2023-03-23 Thread Tsz Wo Sze
solutions are down in this thread -Ayush On Wed, 22 Mar 2023 at 7:48 AM, Tsz Wo Sze wrote: > > Ayush, > > > Yes, reflections are a part of Java.  Why we have to define the > FileSystem APIs but not simply use reflections all the times? > > > Reflection is good for deal

Re: [DISCUSS] Move HDFS specific APIs to FileSystem abstration

2023-03-23 Thread Tsz Wo Sze
stuff for Hbase and then other folks will keep on managing, maintaining and releasing them forever!!! Good Luck!!! But 2 possible solutions are down in this thread -Ayush On Wed, 22 Mar 2023 at 7:48 AM, Tsz Wo Sze wrote: > > Ayush, > > > Yes, reflections are a part of Jav

Re: [DISCUSS] Move HDFS specific APIs to FileSystem abstration

2023-03-21 Thread Tsz Wo Sze
Ayush, Yes, reflections are a part of Java.  Why we have to define the FileSystem APIs but not simply use reflections all the times? Reflection is good for dealing with unknown code such as loading a plugin, code analysis, etc.   However, it probably is not a good way to define APIs.

Re: [DISCUSS] Move HDFS specific APIs to FileSystem abstration

2023-03-20 Thread Tsz Wo Sze
It makes a lot sense to use PathCapabilities. Reflection is a hack but not a solution. Tsz-WoOn Tuesday, March 21, 2023, 09:43:15 AM GMT+8, Ayush Saxena wrote: Hbase doesn’t want to add Ozone as a dependency sounds to me like a ‘Hbase having resistance against the people proposing or

[jira] [Resolved] (HDFS-16885) Fix TestHdfsConfigFields#testCompareConfigurationClassAgainstXml failed

2023-01-09 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-16885. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed The pull request is now

[jira] [Resolved] (HDFS-16881) Warn if AccessControlEnforcer runs for a long time to check permission

2023-01-04 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-16881. --- Fix Version/s: 1.3.0 Hadoop Flags: Reviewed Resolution: Fixed The pull request is now

[jira] [Created] (HDFS-16881) Print a warning if AccessControlEnforcer runs for a long time to check permission

2023-01-02 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-16881: - Summary: Print a warning if AccessControlEnforcer runs for a long time to check permission Key: HDFS-16881 URL: https://issues.apache.org/jira/browse/HDFS-16881 Project

[jira] [Resolved] (HDFS-12006) Ozone: add TestDistributedOzoneVolumesRatis, TestOzoneRestWithMiniClusterRatis and TestOzoneWebAccessRatis

2022-10-15 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-12006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-12006. --- Resolution: Won't Do > Ozone: add TestDistributedOzoneVolumesRa

Re: [VOTE] Moving Ozone to a separated Apache project

2020-09-25 Thread Tsz Wo Sze
+1Tsz-Wo On Saturday, September 26, 2020, 01:42:10 AM GMT+8, anu engineer wrote: +1. --Anu On Thu, Sep 24, 2020 at 10:59 PM Elek, Marton wrote: > Hi all, > > Thank you for all the feedback and requests, > > As we discussed in the previous thread(s) [1], Ozone is proposed to be a >

[jira] [Resolved] (HDFS-15519) Check inaccessible INodes in FsImageValidation

2020-08-14 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-15519. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Thanks, [~shashikant

[jira] [Created] (HDFS-15523) Fix the findbugs warnings from HDFS-15520

2020-08-11 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-15523: - Summary: Fix the findbugs warnings from HDFS-15520 Key: HDFS-15523 URL: https://issues.apache.org/jira/browse/HDFS-15523 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-15521) Remove INode.dumpTreeRecursively()

2020-08-07 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-15521: - Summary: Remove INode.dumpTreeRecursively() Key: HDFS-15521 URL: https://issues.apache.org/jira/browse/HDFS-15521 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-15520) Use visitor pattern to visit namespace tree

2020-08-07 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-15520: - Summary: Use visitor pattern to visit namespace tree Key: HDFS-15520 URL: https://issues.apache.org/jira/browse/HDFS-15520 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-15519) Check inaccessible INodes in FsImageValidation

2020-08-07 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-15519: - Summary: Check inaccessible INodes in FsImageValidation Key: HDFS-15519 URL: https://issues.apache.org/jira/browse/HDFS-15519 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-15483) Ordered snapshot deletion: Disallow rename between two snapshottable directories

2020-07-19 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-15483: - Summary: Ordered snapshot deletion: Disallow rename between two snapshottable directories Key: HDFS-15483 URL: https://issues.apache.org/jira/browse/HDFS-15483 Project

[jira] [Created] (HDFS-15482) Ordered snapshot deletion: hide the deleted snapshots from users

2020-07-19 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-15482: - Summary: Ordered snapshot deletion: hide the deleted snapshots from users Key: HDFS-15482 URL: https://issues.apache.org/jira/browse/HDFS-15482 Project: Hadoop HDFS

[jira] [Created] (HDFS-15481) Ordered snapshot deletion: garbage collect deleted snapshots

2020-07-19 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-15481: - Summary: Ordered snapshot deletion: garbage collect deleted snapshots Key: HDFS-15481 URL: https://issues.apache.org/jira/browse/HDFS-15481 Project: Hadoop HDFS

[jira] [Created] (HDFS-15480) Ordered snapshot deletion: record snapshot deletion in XAttr

2020-07-19 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-15480: - Summary: Ordered snapshot deletion: record snapshot deletion in XAttr Key: HDFS-15480 URL: https://issues.apache.org/jira/browse/HDFS-15480 Project: Hadoop HDFS

[jira] [Created] (HDFS-15479) Add a conf for enforcing ordered snapshot deletion

2020-07-19 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-15479: - Summary: Add a conf for enforcing ordered snapshot deletion Key: HDFS-15479 URL: https://issues.apache.org/jira/browse/HDFS-15479 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-15477) Enforce in-order snapshot deletion

2020-07-18 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-15477: - Summary: Enforce in-order snapshot deletion Key: HDFS-15477 URL: https://issues.apache.org/jira/browse/HDFS-15477 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-15463) Add a tool to validate FsImage

2020-07-09 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDFS-15463: - Summary: Add a tool to validate FsImage Key: HDFS-15463 URL: https://issues.apache.org/jira/browse/HDFS-15463 Project: Hadoop HDFS Issue Type: New Feature

[jira] [Created] (HDDS-2523) BufferPool.releaseBuffer may release a buffer different than the head of the list

2019-11-17 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2523: Summary: BufferPool.releaseBuffer may release a buffer different than the head of the list Key: HDDS-2523 URL: https://issues.apache.org/jira/browse/HDDS-2523 Project

[jira] [Created] (HDDS-2386) Implement incremental ChunkBuffer

2019-10-30 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2386: Summary: Implement incremental ChunkBuffer Key: HDDS-2386 URL: https://issues.apache.org/jira/browse/HDDS-2386 Project: Hadoop Distributed Data Store Issue Type

[jira] [Created] (HDDS-2375) Refactor BlockOutputStream to allow flexible buffering

2019-10-28 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2375: Summary: Refactor BlockOutputStream to allow flexible buffering Key: HDDS-2375 URL: https://issues.apache.org/jira/browse/HDDS-2375 Project: Hadoop Distributed Data Store

[jira] [Created] (HDDS-2275) In BatchOperation.SingleOperation, do not clone byte[]

2019-10-09 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2275: Summary: In BatchOperation.SingleOperation, do not clone byte[] Key: HDDS-2275 URL: https://issues.apache.org/jira/browse/HDDS-2275 Project: Hadoop Distributed Data Store

[jira] [Created] (HDDS-2274) Avoid buffer copying in Codec

2019-10-09 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2274: Summary: Avoid buffer copying in Codec Key: HDDS-2274 URL: https://issues.apache.org/jira/browse/HDDS-2274 Project: Hadoop Distributed Data Store Issue Type

[jira] [Created] (HDDS-2273) Avoid buffer copying in GrpcReplicationService

2019-10-09 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2273: Summary: Avoid buffer copying in GrpcReplicationService Key: HDDS-2273 URL: https://issues.apache.org/jira/browse/HDDS-2273 Project: Hadoop Distributed Data Store

[jira] [Created] (HDDS-2272) Avoid buffer copying in GrpcReplicationClient

2019-10-09 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2272: Summary: Avoid buffer copying in GrpcReplicationClient Key: HDDS-2272 URL: https://issues.apache.org/jira/browse/HDDS-2272 Project: Hadoop Distributed Data Store

[jira] [Created] (HDDS-2271) Avoid buffer copying in KeyValueHandler

2019-10-09 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2271: Summary: Avoid buffer copying in KeyValueHandler Key: HDDS-2271 URL: https://issues.apache.org/jira/browse/HDDS-2271 Project: Hadoop Distributed Data Store Issue

[jira] [Created] (HDDS-2270) Avoid buffer copying in ContainerStateMachine.loadSnapshot/persistContainerSet

2019-10-08 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2270: Summary: Avoid buffer copying in ContainerStateMachine.loadSnapshot/persistContainerSet Key: HDDS-2270 URL: https://issues.apache.org/jira/browse/HDDS-2270 Project: Hadoop

[jira] [Resolved] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-04 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDDS-. -- Resolution: Fixed Merged Pull Request #1595. > Add a method to update ByteBuffer in PureJavaCr

[jira] [Reopened] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-04 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze reopened HDDS-: -- Reopen to change findbugsExcludeFile.xml > Add a method to update ByteBuffer in PureJavaCr

[jira] [Created] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-: Summary: Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C Key: HDDS- URL: https://issues.apache.org/jira/browse/HDDS- Project: Hadoop Distributed

[jira] [Created] (HDDS-2204) Avoid buffer coping in checksum verification

2019-09-29 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2204: Summary: Avoid buffer coping in checksum verification Key: HDDS-2204 URL: https://issues.apache.org/jira/browse/HDDS-2204 Project: Hadoop Distributed Data Store

Re: [EVENT] HDFS Bug Bash: March 12

2018-03-09 Thread Tsz Wo Sze
+1 Thanks for making this happens! Tsz-Wo On Tuesday, March 6, 2018, 7:49:06 PM PST, Chris Douglas wrote: Found a meetup alternative (thanks Subru): https://meetingstar.io/event/fk13172f1d75KN So we can get a rough headcount, please add (local) if you plan to

Re: HDFS Balancer - Balancing Effectively ?

2016-08-11 Thread Tsz Wo Sze
Hi Senthi The Balancer performance was improved dramatically recently [1].  I am not sure if you know about the new conf and parameters; see [2].  If you are interested in more details on how the Balancer works, please see [3].  Thanks. 1. 

Re: [VOTE] HDFS-1312 - Diskbalancer branch merge

2016-06-21 Thread Tsz Wo Sze
+1 The code looks good in general.  It is great that there are a lot of tests and documentation.  Some minor comments which can be addressed after merge:- There are a few TODOs in the code.- Tried the  help command "hdfs diskbalancer -help plan".  There is a typo "wetolerate" in 

Re: [HDFS-9038] Non-Dfs used Calculation

2016-04-13 Thread Tsz Wo Sze
available = usage.getAvailable() - reserved It is incorrect to minus reserved from usage.getAvailable() above since the reserved space, which is the space reserved for non-hdfs used, may already be occupied by some non-hdfs files but not necessarily empty space. In pre HDFS-5215 calculation,

Re: IMPORTANT: automatic changelog creation

2015-04-02 Thread Tsz Wo Sze
Generating change log from JIRA is a good idea.  It bases on an assumption that each JIRA has an accurate summary (a.k.a. JIRA title) to reflect the committed change. Unfortunately, the assumption is invalid for many cases since we never enforce that the JIRA summary must be the same as the

Re: [VOTE] Release Apache Hadoop 2.6.0

2014-11-17 Thread Tsz Wo Sze
I agree that HDFS-6833 is not a blocker.  It is not a problem for the blocks with replication = 3 and it is not a regression (please correct me if I am wrong.)Tsz-Wo On Monday, November 17, 2014 10:06 AM, Suresh Srinivas sur...@hortonworks.com wrote: Tsuyoshi, thanks for bringing

Re: [VOTE] Release Apache Hadoop 2.6.0

2014-11-17 Thread Tsz Wo Sze
Hi Arun, Only md5 is provided but not mds?  Is it intentional? Tsz-Wo On Monday, November 17, 2014 2:14 PM, Tsz Wo Sze szets...@yahoo.com wrote: I agree that HDFS-6833 is not a blocker.  It is not a problem for the blocks with replication = 3 and it is not a regression (please

Re: [VOTE] Release Apache Hadoop 2.6.0

2014-11-17 Thread Tsz Wo Sze
+1 (suppose using md5 is intentional.) Verified md5 and ran a single node HA HDFS cluster for some time.  Everything worked great. Regards,Tsz-Wo On Monday, November 17, 2014 4:48 PM, Tsz Wo Sze szets...@yahoo.com wrote: Hi Arun, Only md5 is provided but not mds

Re: [VOTE] Merge HDFS-6584 Archival Storage to trunk

2014-09-25 Thread Tsz Wo Sze
Hi HDFS dev, Both HDFS-7081 and HDFS-7095 have been completed. I have also merged HDFS-6584 to branch-2. The Archival Storage feature will be available in the coming Hadoop 2.6.0 release. Thanks. Tsz-Wo On Friday, September 19, 2014 10:21 PM, Tsz Wo Sze szets...@yahoo.com wrote: Hi

Re: [VOTE] Merge HDFS-6584 Archival Storage to trunk

2014-09-19 Thread Tsz Wo Sze
the great work! Tsz-Wo On Friday, September 19, 2014 6:19 AM, Tsz Wo Sze szets...@yahoo.com wrote: Hi HDFS dev, We have four +1's (Vinay, Jitendra, Arpit and me) and no -1's. The VOTE passed. PS: Sorry that my email box seems having some problem. I somehow did not receive Arpit's

Re: [VOTE] Merge HDFS-6584 Archival Storage to trunk

2014-09-18 Thread Tsz Wo Sze
. Tsz-Wo On Tuesday, September 16, 2014 11:20 PM, Tsz Wo Sze szets...@yahoo.com wrote: Hi HDFS dev, We have three binding +1's (Vinay, Jitendra and me) and no -1's. The VOTE passed. Thanks everyone. Tsz-Wo On Monday, September 8, 2014 4:03 PM, Nate Payne npa...@cardinalpath.com

[VOTE] Merge HDFS-6584 Archival Storage to trunk

2014-09-07 Thread Tsz Wo Sze
Hi HDFS dev, We would like to propose merging the development branch HDFS-6584 back to trunk. The work of HDFS-6584 is to support Archival Storage in HDFS. Hadoop needs a solution to decouple growing storage capacity from compute capacity. Nodes with higher density and less expensive storage

Re: Putting the hdfs client as a separate jar

2014-04-02 Thread Tsz Wo Sze
It is a very good idea although it might not be easy to do.  One aspect to consider is that do we need separated jars for rpc client and web client?  Now, suppose we could successfully separate HFDS Client jar(s) from HDFS.  However, HDFS Client uses Common as a library.  We have to separate 

Re: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement

2014-03-04 Thread Tsz Wo Sze
Hi hdfs-dev, With ten +1's (eight binding and two non-binding), the vote passed.  Once the latest merge patch has passed Jenkins, I will merge the HDFS-5535 branch to trunk.  Thanks everyone! Tsz-Wo On Tuesday, February 25, 2014 1:41 PM, Tsz Wo Sze szets...@yahoo.com wrote: Hi hdfs-dev

[VOTE] Merge HDFS-5535 Rolling Upgrade Improvement

2014-02-25 Thread Tsz Wo Sze
Hi hdfs-dev, We propose merging the HDFS-5535 branch to trunk. HDFS Rolling Upgrade is a feature to allow upgrading individual HDFS daemons.   In Hadoop v2, HDFS supports highly-available (HA) namenode services and wire compatibility. These two capabilities make it feasible to upgrade HDFS

Re: [VOTE] Merge HDFS-2832 Heterogeneous Storage Phase 1 to trunk

2013-12-03 Thread Tsz Wo Sze
+1 Tsz-Wo On Tuesday, December 3, 2013 10:32 AM, Jun Ping Du j...@vmware.com wrote: +1. Good to see HDFS can support different storage tiers. I have been involved with minor development bug fixing effort and I agree it is ready to merge too. Thanks, Junping - Original Message

Hadoop-Hdfs-Snapshots-Branch-build

2013-07-29 Thread Tsz Wo Sze
Hi, Since the snapshot branch (HDFS-2802) has been merged to the main branch for a few months (http://s.apache.org/9Qi), I am to going to delete the corresponding Jenkins build (https://builds.apache.org/job/Hadoop-Hdfs-Snapshots-Branch-build/) tomorrow.   Please feel free to let me know if

Re: [VOTE] Merge HDFS-2802 snapshot feature to trunk

2013-05-06 Thread Tsz Wo Sze
+1 Tsz-Wo From: Suresh Srinivas sur...@hortonworks.com To: hdfs-dev@hadoop.apache.org hdfs-dev@hadoop.apache.org Sent: Wednesday, May 1, 2013 11:54 AM Subject: [VOTE] Merge HDFS-2802 snapshot feature to trunk This is a follow up to my earlier heads up

Re: Heads up - Snapshots feature merge into trunk

2013-04-18 Thread Tsz Wo Sze
Hi Aaron, Thanks for supporting the snapshot feature. Currently, allowSnapshot(..) and disallowSnapshot(..) are already in HdfsAdmin.  The other operations createSnapshot(..), renameSnapshot(..) and deleteSnapshot(..) are actually user operations and they are declared in FileSystem.  Users

Re: collision in the naming of '.snapshot' directory between hdfs snapshot and hbase snapshot

2013-04-18 Thread Tsz Wo Sze
@HBase-dev, thanks for yielding the reserved word .snapshot to HDFS and the fast fix for addressing the problem.  You guys have done a great job! @Harsh, it seems that more people think that .snapshot is better. Tsz-Wo From: Andrew Purtell

Re: Heads up - Snapshots feature merge into trunk

2013-04-18 Thread Tsz Wo Sze
@hadoop.apache.org; Tsz Wo Sze szets...@yahoo.com Sent: Thursday, April 18, 2013 1:49 PM Subject: Re: Heads up - Snapshots feature merge into trunk On Fri, Apr 19, 2013 at 4:48 AM, Tsz Wo Sze szets...@yahoo.com wrote: Currently, allowSnapshot(..) and disallowSnapshot(..) are already in HdfsAdmin

Re: VOTE: HDFS-347 merge

2013-04-08 Thread Tsz Wo Sze
Hi Colin, Thanks for closing the previous VOTE.  (We usually count the numbers of +1's and -1's, and then state whether the vote passes.  Hope that you could include such information next time.) I withdraw my -1. Tsz-Wo From: Tsz Wo Sze szets

Re: VOTE: HDFS-347 merge

2013-04-06 Thread Tsz Wo Sze
The patch in HDFS-4661 has addressed the problem I raised.  Once the previous VOTE has be concluded, I will remove my -1.  Thanks. Tsz-Wo From: Tsz Wo Sze szets...@yahoo.com To: hdfs-dev@hadoop.apache.org hdfs-dev@hadoop.apache.org Sent: Friday, April 5

Re: VOTE: HDFS-347 merge

2013-04-04 Thread Tsz Wo Sze
Colin, We usually conclude the last VOTE before starting a new one.  Otherwise, people may be confused between the VOTEs.  (In case you don't know our convention.  Please check with someone before starting a VOTE.  Thanks.) -1 * The previous VOTE started by Colin has not been concluded. *

Re: VOTE: HDFS-347 merge

2013-04-03 Thread Tsz Wo Sze
Hi Colin, Thanks for filing HDFS-4661.  Will watch it. For the latest patch in HDFS-347, it seems that there are still bugs in the patch.  I have posted some comments on HDFS-347.  Thanks again. Nicholas From: Colin McCabe cmcc...@alumni.cmu.edu To:

Re: [Vote] Merge branch-trunk-win to trunk

2013-03-25 Thread Tsz Wo Sze
Hi Konstantin S, I accidentally merged some windows bug fixes to branch-2.  However, branch-2 did not compile -- it showed that the windows changes were missing in branch-2.  The ones causing compilation problems were already reverted (Thanks Sid and Suresh.)  Sorry for the inconvenience. 

Re: VOTE: HDFS-347 merge

2013-03-05 Thread Tsz Wo Sze
Hi Colin, It is great to hear that you agree to keep HDFS-2246.  Please as well address my comments posted on HDFS-347 and let me know once you have posted a new patch on HDFS-347.  Thanks a lot! BTW, you should conclude the VOTE you started and use a separated thread for other discussion as

Re: VOTE: HDFS-347 merge

2013-02-25 Thread Tsz Wo Sze
hdfs-dev@hadoop.apache.org; Tsz Wo Sze szets...@yahoo.com Sent: Monday, February 25, 2013 10:24 AM Subject: Re: VOTE: HDFS-347 merge On Sat, Feb 23, 2013 at 4:23 PM, Tsz Wo Sze szets...@yahoo.com wrote: I still do not see a valid reason to remove HDFS-2246 immediately.  Some users may have

Re: VOTE: HDFS-347 merge

2013-02-23 Thread Tsz Wo Sze
From: Aaron T. Myers a...@cloudera.com To: hdfs-dev@hadoop.apache.org hdfs-dev@hadoop.apache.org; Tsz Wo Sze szets...@yahoo.com Sent: Friday, February 22, 2013 6:40 PM Subject: Re: VOTE: HDFS-347 merge On Fri, Feb 22, 2013 at 6:32 PM, Tsz Wo Sze szets...@yahoo.com

Re: VOTE: HDFS-347 merge

2013-02-22 Thread Tsz Wo Sze
Eli, please see my earlier response below. On Wed, Feb 20, 2013 at 5:12 PM, Aaron T. Myers a...@cloudera.com wrote: Given that the only substantive concerns with HDFS-347 seem to be about Windows support for local reads, for now we only merge this branch to trunk. Another substantive

Re: VOTE: HDFS-347 merge

2013-02-21 Thread Tsz Wo Sze
On Wed, Feb 20, 2013 at 5:12 PM, Aaron T. Myers a...@cloudera.com wrote: Given that the only substantive concerns with HDFS-347 seem to be about Windows support for local reads, for now we only merge this branch to trunk. Another substantive concern is that HDFS-347 is not as well tested as

Re: VOTE: HDFS-347 merge

2013-02-20 Thread Tsz Wo Sze
-1 The patch seems not ready yet.  I have posted some comments/suggestions on the JIRA.  Colin also has agreed that there are some bugs to be fixed.  Sorry. Tsz-Wo From: Todd Lipcon t...@cloudera.com To: hdfs-dev@hadoop.apache.org Sent: Tuesday, February

Re: VOTE: HDFS-347 merge

2013-02-20 Thread Tsz Wo Sze
From: Todd Lipcon t...@cloudera.com To: hdfs-dev@hadoop.apache.org; Tsz Wo Sze szets...@yahoo.com Sent: Wednesday, February 20, 2013 12:16 PM Subject: Re: VOTE: HDFS-347 merge Hi Nicholas, I looked at your comments on the JIRA, and they all seem like trivial things that could be addressed post

Re: VOTE: HDFS-347 merge

2013-02-20 Thread Tsz Wo Sze
Also, the patch seems to have removed the existing short-circuit read feature (HDFS-2246).  It is an incompatible change.  I think the patch is farther away from being ready and I would keep my -1. Tsz-Wo From: Tsz Wo Sze szets...@yahoo.com To: hdfs-dev

Re: VOTE: HDFS-347 merge

2013-02-20 Thread Tsz Wo Sze
The reason to keep it around is that the HDFS-347 only support Unix but not other OS. Tsz-Wo From: Todd Lipcon t...@cloudera.com To: hdfs-dev@hadoop.apache.org; Tsz Wo Sze szets...@yahoo.com Sent: Wednesday, February 20, 2013 3:06 PM Subject: Re: VOTE: HDFS

Re: VOTE: HDFS-347 merge

2013-02-18 Thread Tsz Wo Sze
Hi Colin, The latest HDFS-347 patch was posted on Feb 16.  Because of the long weekends, I still do not have a chance to check it.  Will do it in this week. Nicholas From: Colin McCabe cmcc...@alumni.cmu.edu To: hdfs-dev@hadoop.apache.org Sent: Sunday,

Re: hsync in hadoop 2.0?

2012-06-04 Thread Tsz Wo Sze
+1 Since no one is objecting, I will merge HDFS-744 to 2.0. For 1.x, it needs more thoughts because it is likely to be an incompatible change. Tsz-Wo From: Todd Lipcon t...@cloudera.com To: hdfs-dev@hadoop.apache.org Cc: lars hofhansl lhofha...@yahoo.com

Re: [DISCUSS] Remove append?

2012-03-26 Thread Tsz Wo Sze
Just one comment: If we do decide to keep append in, we should get it to be actually stable and usable.  In my opinion, this should definitely happen before adding any new operations. @Colin, append is currently stable and, of course, usable.  Many people in different organizations have

Re: [DISCUSS] Remove append?

2012-03-26 Thread Tsz Wo Sze
Hi Colin, Please feel free to file JIRAs if you see unit test failures. Let's continue the immutable file discussion on HDFS-3154. Nicholas From: Colin McCabe cmcc...@alumni.cmu.edu To: hdfs-dev@hadoop.apache.org; Tsz Wo Sze szets...@yahoo.com Sent: Monday

Re: svn commit: r1304067 - in /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project: ./ bin/ conf/ hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/ hadoop-mapreduce-exam

2012-03-23 Thread Tsz Wo Sze
are that sub-directory. Regards, Nicholas - Original Message - From: Harsh J ha...@cloudera.com To: mapreduce-...@hadoop.apache.org; Tsz Wo Sze szets...@yahoo.com Cc: hdfs dev hdfs-dev@hadoop.apache.org Sent: Thursday, March 22, 2012 9:45 PM Subject: Re: svn commit: r1304067 - in /hadoop/common

Re: svn commit: r1304067 - in /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project: ./ bin/ conf/ hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/ hadoop-mapreduce-exam

2012-03-22 Thread Tsz Wo Sze
Hi Eli, For merging a HDFS issue, please don't merge COMMON and MAPREDUCE.  It generates useless merge info and emails.  Does it make sense? Regards, Nicholas From: e...@apache.org e...@apache.org To: mapreduce-comm...@hadoop.apache.org Sent: Thursday,

  1   2   >