[jira] [Commented] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-26 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220215#comment-16220215 ] Ewan Higgs commented on HDFS-11902: --- +1. The new patch looks good. The only difference is that code is

[jira] [Assigned] (HDFS-12713) [READ] Refactor FileRegion and BlockAliasMap to separate out HDFS metadata and PROVIDED storage metadata

2017-10-26 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs reassigned HDFS-12713: - Assignee: Ewan Higgs > [READ] Refactor FileRegion and BlockAliasMap to separate out HDFS

[jira] [Assigned] (HDFS-9806) Allow HDFS block replicas to be provided by an external storage system

2017-10-26 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs reassigned HDFS-9806: Assignee: Ewan Higgs > Allow HDFS block replicas to be provided by an external storage system >

[jira] [Assigned] (HDFS-9806) Allow HDFS block replicas to be provided by an external storage system

2017-10-26 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs reassigned HDFS-9806: Assignee: (was: Ewan Higgs) > Allow HDFS block replicas to be provided by an external storage

[jira] [Commented] (HDFS-12594) SnapshotDiff - snapshotDiff fails if the snapshotDiff report exceeds the RPC response limit

2017-10-24 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216518#comment-16216518 ] Ewan Higgs commented on HDFS-12594: --- Some minor things on a first pass: {code} + if

[jira] [Commented] (HDFS-12665) [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb)

2017-10-21 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16214114#comment-16214114 ] Ewan Higgs commented on HDFS-12665: --- [~virajith], thanks for the quick review. {quote}Can you please

[jira] [Commented] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat

2017-10-20 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212698#comment-16212698 ] Ewan Higgs commented on HDFS-12591: --- This now depends on HDFS-12665 since it uses the same code for

[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat

2017-10-20 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12591: -- Attachment: HDFS-12591-HDFS-9806.003.patch Attaching a patch rebased on HDFS-12665 so the two LevelDB

[jira] [Updated] (HDFS-12665) [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb)

2017-10-20 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12665: -- Attachment: HDFS-12665-HDFS-9806.002.patch Attaching an updated version using {{LevelDB}} (instead of

[jira] [Commented] (HDFS-9810) Allow support for more than one block replica per datanode

2017-10-20 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212300#comment-16212300 ] Ewan Higgs commented on HDFS-9810: -- This should be fixed by HDFS-12685. > Allow support for more than one

[jira] [Updated] (HDFS-12665) [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb)

2017-10-19 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12665: -- Parent Issue: HDFS-9806 (was: HDFS-12090) > [AliasMap] Create a version of the AliasMap that runs in

[jira] [Resolved] (HDFS-9808) Combine READ_ONLY_SHARED DatanodeStorages with the same ID

2017-10-19 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs resolved HDFS-9808. -- Resolution: Won't Fix This was part of HDFS-11190. > Combine READ_ONLY_SHARED DatanodeStorages with the

[jira] [Updated] (HDFS-12665) [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb)

2017-10-19 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12665: -- Status: Patch Available (was: Open) > [AliasMap] Create a version of the AliasMap that runs in memory

[jira] [Updated] (HDFS-12665) [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb)

2017-10-19 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12665: -- Attachment: HDFS-12665-HDFS-9806.001.patch Attaching work from WDC implementing the In Memory AliasMap.

[jira] [Assigned] (HDFS-12665) [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb)

2017-10-19 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs reassigned HDFS-12665: - Assignee: Ewan Higgs > [AliasMap] Create a version of the AliasMap that runs in memory in the >

[jira] [Updated] (HDFS-12685) [READ] FsVolumeImpl exception when scanning Provided storage volume

2017-10-19 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12685: -- Description: I left a Datanode running overnight and found this in the logs in the morning: {code}

[jira] [Updated] (HDFS-12685) [READ] FsVolumeImpl exception when scanning Provided storage volume

2017-10-19 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12685: -- Summary: [READ] FsVolumeImpl exception when scanning Provided storage volume (was: FsVolumeImpl

[jira] [Created] (HDFS-12685) FsVolumeImpl exception when scanning Provided storage volume

2017-10-19 Thread Ewan Higgs (JIRA)
Ewan Higgs created HDFS-12685: - Summary: FsVolumeImpl exception when scanning Provided storage volume Key: HDFS-12685 URL: https://issues.apache.org/jira/browse/HDFS-12685 Project: Hadoop HDFS

[jira] [Commented] (HDFS-12605) [READ] TestNameNodeProvidedImplementation#testProvidedDatanodeFailures fails after rebase

2017-10-18 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209894#comment-16209894 ] Ewan Higgs commented on HDFS-12605: --- I tested this with a NN and DN on a local system and it worked. I

[jira] [Issue Comment Deleted] (HDFS-12605) [READ] TestNameNodeProvidedImplementation#testProvidedDatanodeFailures fails after rebase

2017-10-18 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12605: -- Comment: was deleted (was: +1 I'm running into this on the branch and this should fix the issue.) >

[jira] [Commented] (HDFS-12605) [READ] TestNameNodeProvidedImplementation#testProvidedDatanodeFailures fails after rebase

2017-10-18 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209864#comment-16209864 ] Ewan Higgs commented on HDFS-12605: --- +1 I'm running into this on the branch and this should fix the

[jira] [Updated] (HDFS-12090) Handling writes from HDFS to Provided storages

2017-10-18 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12090: -- Attachment: HDFS-12090-Functional-Specification.002.pdf Attaching updated version of Functional

[jira] [Issue Comment Deleted] (HDFS-12090) Handling writes from HDFS to Provided storages

2017-10-17 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12090: -- Comment: was deleted (was: Attaching Functional Specification with description of expected command

[jira] [Updated] (HDFS-12090) Handling writes from HDFS to Provided storages

2017-10-17 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12090: -- Attachment: HDFS-12090-Functional-Specification.001.pdf Attaching functional specification. This should

[jira] [Updated] (HDFS-12090) Handling writes from HDFS to Provided storages

2017-10-17 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12090: -- Attachment: HDFS-12090 Functional Specification.pdf Attaching Functional Specification with description

[jira] [Updated] (HDFS-12090) Handling writes from HDFS to Provided storages

2017-10-17 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12090: -- Attachment: (was: HDFS-12090 Functional Specification.pdf) > Handling writes from HDFS to Provided

[jira] [Created] (HDFS-12666) Provided Storage Mount Manager (PSMM) mount

2017-10-16 Thread Ewan Higgs (JIRA)
Ewan Higgs created HDFS-12666: - Summary: Provided Storage Mount Manager (PSMM) mount Key: HDFS-12666 URL: https://issues.apache.org/jira/browse/HDFS-12666 Project: Hadoop HDFS Issue Type:

[jira] [Created] (HDFS-12665) [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb)

2017-10-16 Thread Ewan Higgs (JIRA)
Ewan Higgs created HDFS-12665: - Summary: [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb) Key: HDFS-12665 URL: https://issues.apache.org/jira/browse/HDFS-12665

[jira] [Updated] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-16 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11902: -- Status: Open (was: Patch Available) > [READ] Merge BlockFormatProvider and FileRegionProvider. >

[jira] [Updated] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-16 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11902: -- Status: Patch Available (was: Open) > [READ] Merge BlockFormatProvider and FileRegionProvider. >

[jira] [Commented] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-16 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205819#comment-16205819 ] Ewan Higgs commented on HDFS-11902: --- I removed patch 10 as it included errors and was only intended to

[jira] [Updated] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-16 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11902: -- Attachment: (was: HDFS-11902-HDFS-9806.010.patch) > [READ] Merge BlockFormatProvider and

[jira] [Updated] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-13 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11902: -- Attachment: HDFS-11902-HDFS-9806.010.patch Patch fixing findbugs issue and making

[jira] [Assigned] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat

2017-10-12 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs reassigned HDFS-12591: - Assignee: Ewan Higgs > [READ] Implement LevelDBFileRegionFormat >

[jira] [Updated] (HDFS-12478) [WRITE] Command line tools for managing Provided Storage Backup mounts

2017-10-11 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12478: -- Status: Patch Available (was: Open) > [WRITE] Command line tools for managing Provided Storage Backup

[jira] [Updated] (HDFS-12478) [WRITE] Command line tools for managing Provided Storage Backup mounts

2017-10-11 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12478: -- Attachment: HDFS-12478-HDFS-9806.001.patch Attaching a patch that implements the command line

[jira] [Commented] (HDFS-11045) TestDirectoryScanner#testThrottling fails: Throttle is too permissive

2017-10-10 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198703#comment-16198703 ] Ewan Higgs commented on HDFS-11045: --- [~dan...@cloudera.com], can you rebase this? {quote} Karthik

[jira] [Commented] (HDFS-11707) TestDirectoryScanner#testThrottling fails on OSX

2017-10-10 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198685#comment-16198685 ] Ewan Higgs commented on HDFS-11707: --- This should probably be closed as duplicate of HDFS-11045 >

[jira] [Commented] (HDFS-11707) TestDirectoryScanner#testThrottling fails on OSX

2017-10-10 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198672#comment-16198672 ] Ewan Higgs commented on HDFS-11707: --- I've seen this error pop up. One theory is that if the throttle

[jira] [Comment Edited] (HDFS-11707) TestDirectoryScanner#testThrottling fails on OSX

2017-10-10 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198672#comment-16198672 ] Ewan Higgs edited comment on HDFS-11707 at 10/10/17 1:33 PM: - I've seen this

[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat

2017-10-04 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12591: -- Attachment: HDFS-12591-HDFS-9806.001.patch Attaching work I've previously done for this. It needs to be

[jira] [Updated] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat

2017-10-04 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12591: -- Status: Patch Available (was: Open) > [READ] Implement LevelDBFileRegionFormat >

[jira] [Created] (HDFS-12591) [READ] Implement LevelDBFileRegionFormat

2017-10-04 Thread Ewan Higgs (JIRA)
Ewan Higgs created HDFS-12591: - Summary: [READ] Implement LevelDBFileRegionFormat Key: HDFS-12591 URL: https://issues.apache.org/jira/browse/HDFS-12591 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Commented] (HDFS-12589) [DISCUSS] Provided Storage BlockAlias Refactoring

2017-10-04 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16191054#comment-16191054 ] Ewan Higgs commented on HDFS-12589: --- Some discussion happened off jira; but we'd much prefer these

[jira] [Created] (HDFS-12589) [DISCUSS] Provided Storage BlockAlias Refactoring

2017-10-04 Thread Ewan Higgs (JIRA)
Ewan Higgs created HDFS-12589: - Summary: [DISCUSS] Provided Storage BlockAlias Refactoring Key: HDFS-12589 URL: https://issues.apache.org/jira/browse/HDFS-12589 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-12478) [WRITE] Command line tools for managing Provided Storage Backup mounts

2017-09-18 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12478: -- Issue Type: Sub-task (was: Task) Parent: HDFS-12090 > [WRITE] Command line tools for managing

[jira] [Assigned] (HDFS-12478) [WRITE] Command line tools for managing Provided Storage Backup mounts

2017-09-18 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs reassigned HDFS-12478: - Assignee: Ewan Higgs > [WRITE] Command line tools for managing Provided Storage Backup mounts >

[jira] [Assigned] (HDFS-12090) Handling writes from HDFS to Provided storages

2017-09-18 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs reassigned HDFS-12090: - Assignee: (was: Ewan Higgs) > Handling writes from HDFS to Provided storages >

[jira] [Assigned] (HDFS-12090) Handling writes from HDFS to Provided storages

2017-09-18 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs reassigned HDFS-12090: - Assignee: Ewan Higgs > Handling writes from HDFS to Provided storages >

[jira] [Created] (HDFS-12478) [WRITE] Command line tools for managing Provided Storage Backup mounts

2017-09-18 Thread Ewan Higgs (JIRA)
Ewan Higgs created HDFS-12478: - Summary: [WRITE] Command line tools for managing Provided Storage Backup mounts Key: HDFS-12478 URL: https://issues.apache.org/jira/browse/HDFS-12478 Project: Hadoop HDFS

[jira] [Commented] (HDFS-12344) LocatedFileStatus regression: no longer accepting null FSPermission

2017-08-24 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139971#comment-16139971 ] Ewan Higgs commented on HDFS-12344: --- javac issue is because i introduced a new call to a deprecated

[jira] [Updated] (HDFS-12344) LocatedFileStatus regression: no longer accepting null FSPermission

2017-08-24 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12344: -- Attachment: HDFS-12344.002.patch Attaching updated patch based on [~steve_l]'s comments. >

[jira] [Updated] (HDFS-12344) LocatedFileStatus regression: no longer accepting null FSPermission

2017-08-24 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12344: -- Status: Patch Available (was: Open) > LocatedFileStatus regression: no longer accepting null

[jira] [Updated] (HDFS-12344) LocatedFileStatus regression: no longer accepting null FSPermission

2017-08-24 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12344: -- Attachment: HDFS-12344.001.patch Attaching a failing test and the fix. [~steve_l] and/or

[jira] [Assigned] (HDFS-12344) LocatedFileStatus regression: no longer accepting null FSPermission

2017-08-23 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs reassigned HDFS-12344: - Assignee: Ewan Higgs > LocatedFileStatus regression: no longer accepting null FSPermission >

[jira] [Created] (HDFS-12344) LocatedFileStatus regression: no longer accepting null FSPermission

2017-08-23 Thread Ewan Higgs (JIRA)
Ewan Higgs created HDFS-12344: - Summary: LocatedFileStatus regression: no longer accepting null FSPermission Key: HDFS-12344 URL: https://issues.apache.org/jira/browse/HDFS-12344 Project: Hadoop HDFS

[jira] [Commented] (HDFS-11639) [READ] Encode the BlockAlias in the client protocol

2017-08-23 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138036#comment-16138036 ] Ewan Higgs commented on HDFS-11639: --- {quote}Hi Ewan Higgs, As this change is required only for writes,

[jira] [Updated] (HDFS-11639) [READ] Encode the BlockAlias in the client protocol

2017-08-23 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11639: -- Parent Issue: HDFS-12090 (was: HDFS-9806) > [READ] Encode the BlockAlias in the client protocol >

[jira] [Commented] (HDFS-11828) [READ] Refactor FsDatasetImpl to use the BlockAlias from ClientProtocol for PROVIDED blocks.

2017-08-23 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138040#comment-16138040 ] Ewan Higgs commented on HDFS-11828: --- {quote} Ewan Higgs, Similar to HDFS-11639, we can make this a

[jira] [Updated] (HDFS-11828) [READ] Refactor FsDatasetImpl to use the BlockAlias from ClientProtocol for PROVIDED blocks.

2017-08-23 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11828: -- Parent Issue: HDFS-12090 (was: HDFS-9806) > [READ] Refactor FsDatasetImpl to use the BlockAlias from

[jira] [Commented] (HDFS-12289) [READ] HDFS-12091 breaks the tests for provided block reads

2017-08-11 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124220#comment-16124220 ] Ewan Higgs commented on HDFS-12289: --- LGTM > [READ] HDFS-12091 breaks the tests for provided block reads

[jira] [Commented] (HDFS-12091) [READ] Check that the replicas served from a {{ProvidedVolumeImpl}} belong to the correct external storage

2017-08-02 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1687#comment-1687 ] Ewan Higgs commented on HDFS-12091: --- Thanks for adding the test. I tested the first patch and now we

[jira] [Commented] (HDFS-12091) [READ] Check that the replicas served from a {{ProvidedVolumeImpl}} belong to the correct external storage

2017-07-27 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103235#comment-16103235 ] Ewan Higgs commented on HDFS-12091: --- I tested it on a simple 1NN 1DN setup and it worked. Could you add

[jira] [Comment Edited] (HDFS-12093) [READ] Share remoteFS between ProvidedReplica instances.

2017-07-27 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103078#comment-16103078 ] Ewan Higgs edited comment on HDFS-12093 at 7/27/17 12:09 PM: - Tested this on a

[jira] [Commented] (HDFS-12093) [READ] Share remoteFS between ProvidedReplica instances.

2017-07-27 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103078#comment-16103078 ] Ewan Higgs commented on HDFS-12093: --- Tested this on a simple 1 NN 1 DN shared machine and it was able to

[jira] [Commented] (HDFS-12151) Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes

2017-07-26 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101347#comment-16101347 ] Ewan Higgs commented on HDFS-12151: --- {quote}Should we use {{nst > 0}} rather than

[jira] [Commented] (HDFS-12151) Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes

2017-07-26 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101320#comment-16101320 ] Ewan Higgs commented on HDFS-12151: --- +1. Don't know how I missed that in the fix for HDFS-11956.

[jira] [Created] (HDFS-12093) [READ] Share remoteFS between ProvidedReplica instances.

2017-07-06 Thread Ewan Higgs (JIRA)
Ewan Higgs created HDFS-12093: - Summary: [READ] Share remoteFS between ProvidedReplica instances. Key: HDFS-12093 URL: https://issues.apache.org/jira/browse/HDFS-12093 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients

2017-06-26 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11956: -- Attachment: HDFS-11956.004.patch Attaching version of the patch that doesn't use a config switch. >

[jira] [Commented] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients

2017-06-22 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16060089#comment-16060089 ] Ewan Higgs commented on HDFS-11956: --- Hi Andrew {quote} IIUC, we know the storage type even for an old

[jira] [Commented] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients

2017-06-22 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16059663#comment-16059663 ] Ewan Higgs commented on HDFS-11956: --- Hi, Another idea is to just ignore the BlockTokenIdentifier if the

[jira] [Commented] (HDFS-11125) [SPS]: Use smaller batches of BlockMovingInfo into the block storage movement command

2017-06-20 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16056712#comment-16056712 ] Ewan Higgs commented on HDFS-11125: --- The Provided Storage system is looking to use the SPS for

[jira] [Updated] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients

2017-06-15 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11956: -- Attachment: HDFS-11956.003.patch Attaching patch with value for hdfs-default.xml and some checkstyle

[jira] [Updated] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients

2017-06-14 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11956: -- Attachment: HDFS-11956.002.patch Attaching updated patch with a unit test. In the test, {{strictSM}}

[jira] [Updated] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients

2017-06-14 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11956: -- Assignee: Ewan Higgs (was: Chris Douglas) Release Note: Introduce

[jira] [Updated] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients

2017-06-14 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11956: -- Attachment: HDFS-11956.001.patch Attaching a patch that introduces

[jira] [Commented] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients

2017-06-14 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049350#comment-16049350 ] Ewan Higgs commented on HDFS-11956: --- I took a look and see that this fails when writing blocks. e.g.:

[jira] [Commented] (HDFS-11640) [READ] Datanodes should use a unique identifier when reading from external stores

2017-05-23 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020979#comment-16020979 ] Ewan Higgs commented on HDFS-11640: --- I ran into compilation errors when trying to build this. I suppose

[jira] [Updated] (HDFS-11639) [READ] Encode the BlockAlias in the client protocol

2017-05-23 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11639: -- Attachment: HDFS-11639-HDFS-9806.005.patch Attaching a patch that removes the {{BlockAlias}} from the

[jira] [Commented] (HDFS-11639) [READ] Encode the BlockAlias in the client protocol

2017-05-18 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16015479#comment-16015479 ] Ewan Higgs commented on HDFS-11639: --- {quote}Any particular reason for changing BlockProvider to

[jira] [Updated] (HDFS-11639) [READ] Encode the BlockAlias in the client protocol

2017-05-17 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11639: -- Status: Patch Available (was: Open) > [READ] Encode the BlockAlias in the client protocol >

[jira] [Updated] (HDFS-11639) [READ] Encode the BlockAlias in the client protocol

2017-05-17 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11639: -- Attachment: HDFS-11639-HDFS-9806.003.patch Attaching an updated patch that addresses some of

[jira] [Created] (HDFS-11828) Refactor FsDatasetImpl as the BlockAlias is in the wire protocol for PROVIDED blocks.

2017-05-16 Thread Ewan Higgs (JIRA)
Ewan Higgs created HDFS-11828: - Summary: Refactor FsDatasetImpl as the BlockAlias is in the wire protocol for PROVIDED blocks. Key: HDFS-11828 URL: https://issues.apache.org/jira/browse/HDFS-11828

[jira] [Comment Edited] (HDFS-11639) [READ] Encode the BlockAlias in the client protocol

2017-05-15 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010483#comment-16010483 ] Ewan Higgs edited comment on HDFS-11639 at 5/15/17 1:16 PM: {quote} Btw, I

[jira] [Commented] (HDFS-11639) [READ] Encode the BlockAlias in the client protocol

2017-05-15 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010483#comment-16010483 ] Ewan Higgs commented on HDFS-11639: --- {quote} Btw, I rebased the HDFS-9806 branch on the most recent

[jira] [Commented] (HDFS-11639) [READ] Encode the BlockAlias in the client protocol

2017-05-11 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006131#comment-16006131 ] Ewan Higgs commented on HDFS-11639: --- I forgot to mention that the current patch requires

[jira] [Updated] (HDFS-11639) [READ] Encode the BlockAlias in the client protocol

2017-05-10 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11639: -- Attachment: HDFS-11639-HDFS-9806.002.patch Attaching a minor fix where a RuntimeError escaped to break

[jira] [Comment Edited] (HDFS-11639) [READ] Encode the BlockAlias in the client protocol

2017-05-10 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004653#comment-16004653 ] Ewan Higgs edited comment on HDFS-11639 at 5/10/17 1:14 PM: Attached a patch

[jira] [Updated] (HDFS-11639) [READ] Encode the BlockAlias in the client protocol

2017-05-10 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11639: -- Attachment: HDFS-11639-HDFS-9806.001.patch Attached a patch that encodes the {{BlockAlias}} into the

[jira] [Commented] (HDFS-9807) Add an optional StorageID to writes

2017-05-04 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997509#comment-15997509 ] Ewan Higgs commented on HDFS-9807: -- {quote}Posted a new patch (v010) that reverts the unrelated changes to

[jira] [Commented] (HDFS-11703) [READ] Tests for ProvidedStorageMap

2017-05-04 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996651#comment-15996651 ] Ewan Higgs commented on HDFS-11703: --- +1. > [READ] Tests for ProvidedStorageMap >

[jira] [Issue Comment Deleted] (HDFS-11673) [READ] Handle failures of Datanodes with PROVIDED storage

2017-05-04 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11673: -- Comment: was deleted (was: I reproduced the patch problem that jenkins had: {code} $ patch -p1

[jira] [Commented] (HDFS-11673) [READ] Handle failures of Datanodes with PROVIDED storage

2017-05-04 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996596#comment-15996596 ] Ewan Higgs commented on HDFS-11673: --- I reproduced the patch problem that jenkins had: {code} $ patch

[jira] [Commented] (HDFS-11663) [READ] Fix NullPointerException in ProvidedBlocksBuilder

2017-05-04 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996592#comment-15996592 ] Ewan Higgs commented on HDFS-11663: --- +1. Uncontroversial bug fix. > [READ] Fix NullPointerException in

[jira] [Commented] (HDFS-11653) [READ] ProvidedReplica should return an InputStream that is bounded by its length

2017-05-04 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996410#comment-15996410 ] Ewan Higgs commented on HDFS-11653: --- This is a straightforward fix with a unit test. +1 > [READ]

[jira] [Issue Comment Deleted] (HDFS-9807) Add an optional StorageID to writes

2017-05-04 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-9807: - Comment: was deleted (was: {quote}The changes to Host2NodesMap and BlockPlacementPolicyDefault are

[jira] [Commented] (HDFS-9807) Add an optional StorageID to writes

2017-05-04 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996319#comment-15996319 ] Ewan Higgs commented on HDFS-9807: -- {quote}The changes to Host2NodesMap and BlockPlacementPolicyDefault

[jira] [Commented] (HDFS-9807) Add an optional StorageID to writes

2017-05-02 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992494#comment-15992494 ] Ewan Higgs commented on HDFS-9807: -- Thanks [~virajith]! This is good stuff. In your solution using

[jira] [Updated] (HDFS-9807) Add an optional StorageID to writes

2017-04-28 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-9807: - Attachment: HDFS-9807.007.patch Adding a patch that fixes check-style issues and adds a nose-tail test

[jira] [Updated] (HDFS-9807) Add an optional StorageID to writes

2017-04-27 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-9807: - Attachment: HDFS-9807.006.patch Attaching patch rebased now that HDFS-6708 has been merged. > Add an

<    1   2   3   4   5   >