[jira] [Updated] (HDFS-2287) TestParallelRead has a small off-by-one bug
[ https://issues.apache.org/jira/browse/HDFS-2287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2287: Fix Version/s: (was: 0.24.0) TestParallelRead has a small off-by-one bug --- Key: HDFS-2287 URL: https://issues.apache.org/jira/browse/HDFS-2287 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Trivial Fix For: 0.22.0 Attachments: hdfs-2287-v-0.22.patch, hdfs-2287.txt Noticed this bug when I was running TestParallelRead - a simple off-by-one error in some internal bounds checking. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3131) Improve TestStorageRestore
[ https://issues.apache.org/jira/browse/HDFS-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-3131: Target Version/s: 0.24.0, 1.1.0 (was: 1.1.0, 0.24.0) Fix Version/s: (was: 0.24.0) Improve TestStorageRestore -- Key: HDFS-3131 URL: https://issues.apache.org/jira/browse/HDFS-3131 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.24.0, 1.1.0 Reporter: Tsz Wo (Nicholas), SZE Assignee: Brandon Li Priority: Minor Labels: newbie Fix For: 1.1.0 Attachments: HDFS-3131.branch-1.patch, HDFS-3131.patch, HDFS-3131.patch Aaron has the following comments on TestStorageRestore in HDFS-3127. # removeStorageAccess, restoreAccess, and numStorageDirs can all be made private # numStorageDirs can be made static # Rather than do set(Readable/Executable/Writable), use FileUtil.chmod(...). # Please put the contents of the test in a try/finally, with the calls to shutdown the cluster and the 2NN in the finally block. # Some lines are over 80 chars. # No need for the numDatanodes variable - it's only used in one place. # Instead of xwr use rwx, which I think is a more common way of describing permissions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3101) cannot read empty file using webhdfs
[
https://issues.apache.org/jira/browse/HDFS-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated HDFS-3101:
Fix Version/s: (was: 0.23.3)
2.0.0
cannot read empty file using webhdfs
Key: HDFS-3101
URL: https://issues.apache.org/jira/browse/HDFS-3101
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.23.1
Reporter: Zhanwei.Wang
Assignee: Tsz Wo (Nicholas), SZE
Fix For: 0.23.2, 1.0.2, 2.0.0
Attachments: h3101_20120315.patch, h3101_20120315_branch-1.patch
STEP:
1, create a new EMPTY file
2, read it using webhdfs.
RESULT:
expected: get a empty file
I got:
{RemoteException:{exception:IOException,javaClassName:java.io.IOException,message:Offset=0
out of the range [0, 0); OPEN, path=/testFile}}
First of all, [0, 0) is not a valid range, and I think read a empty file
should be OK.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3129) NetworkTopology: add test that getLeaf should check for invalid topologies
[ https://issues.apache.org/jira/browse/HDFS-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-3129: Fix Version/s: (was: 0.23.3) 2.0.0 NetworkTopology: add test that getLeaf should check for invalid topologies -- Key: HDFS-3129 URL: https://issues.apache.org/jira/browse/HDFS-3129 Project: Hadoop HDFS Issue Type: Test Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Priority: Minor Fix For: 1.1.0, 2.0.0 Attachments: HDFS-3129-b1.001.patch, HDFS-3129.001.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2978) The NameNode should expose name dir statuses via JMX
[ https://issues.apache.org/jira/browse/HDFS-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2978: Target Version/s: 1.0.2, 0.23.3 (was: 0.23.3, 1.0.2) Fix Version/s: (was: 0.23.3) 2.0.0 The NameNode should expose name dir statuses via JMX Key: HDFS-2978 URL: https://issues.apache.org/jira/browse/HDFS-2978 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.23.0, 1.0.0 Reporter: Aaron T. Myers Assignee: Aaron T. Myers Fix For: 1.0.2, 2.0.0 Attachments: HDFS-2978-branch-1.patch, HDFS-2978.patch, HDFS-2978.patch We currently display this info on the NN web UI, so users who wish to monitor this must either do it manually or parse HTML. We should publish this information via JMX. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2492) BlockManager cross-rack replication checks only work for ScriptBasedMapping
[
https://issues.apache.org/jira/browse/HDFS-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated HDFS-2492:
Fix Version/s: (was: 0.23.1)
0.23.3
BlockManager cross-rack replication checks only work for ScriptBasedMapping
---
Key: HDFS-2492
URL: https://issues.apache.org/jira/browse/HDFS-2492
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.23.0, 0.24.0
Reporter: Steve Loughran
Assignee: Steve Loughran
Priority: Minor
Fix For: 0.24.0, 0.23.3
Attachments: HDFS-2492-blockmanager.patch,
HDFS-2492-blockmanager.patch, HDFS-2492-blockmanager.patch,
HDFS-2492-blockmanager.patch, HDFS-2492-blockmanager.patch
The BlockManager cross-rack replication checks only works if script files are
used for replication, not if alternate plugins provide the topology
information.
This is because the BlockManager sets its rack checking flag if there is a
filename key
{code}
shouldCheckForEnoughRacks =
conf.get(DFSConfigKeys.NET_TOPOLOGY_SCRIPT_FILE_NAME_KEY) != null;
{code}
yet this filename key is only used if the topology mapper defined by
{code}
DFSConfigKeys.NET_TOPOLOGY_NODE_SWITCH_MAPPING_IMPL_KEY
{code}
is an instance of {{ScriptBasedMapping}}
If any other mapper is used, the system may be multi rack, but the Block
Manager will not be aware of this fact unless the filename key is set to
something non-null
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2696) fuse-dfs build problems
[ https://issues.apache.org/jira/browse/HDFS-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2696: Fix Version/s: (was: 0.23.1) 0.23.3 fuse-dfs build problems --- Key: HDFS-2696 URL: https://issues.apache.org/jira/browse/HDFS-2696 Project: Hadoop HDFS Issue Type: Bug Components: contrib/fuse-dfs Affects Versions: 0.23.1 Environment: linux Reporter: Petru Dimulescu Labels: build, maven, patch Fix For: 0.23.3 Attachments: dfsfuse-build.patch.zip fuse-dfs has some compilation problems on the 0.23 and 0.24(head) branches. Some details on how to compile fuse-dfs on 0.23 is here : http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/201112.mbox/%3c4ee23195.7030...@gmail.com%3E and a wiki page follows. The attached patch removes some problems (an inexistnant ivy/library.properties, detecting jni.h location). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2607) Use named daemon threads for the directory scanner
[ https://issues.apache.org/jira/browse/HDFS-2607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2607: Fix Version/s: (was: 0.23.1) 0.23.3 Use named daemon threads for the directory scanner -- Key: HDFS-2607 URL: https://issues.apache.org/jira/browse/HDFS-2607 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.21.0, 0.22.0, 0.23.0 Reporter: Eli Collins Fix For: 0.23.3 HDFS-854 added a thread pool for block scanners. It would be better to use a factory that names the threads and daemonizes them so they don't block shutdown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2833) Add GETMERGE operation to httpfs
[ https://issues.apache.org/jira/browse/HDFS-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2833: Fix Version/s: (was: 0.23.1) 0.23.3 Add GETMERGE operation to httpfs Key: HDFS-2833 URL: https://issues.apache.org/jira/browse/HDFS-2833 Project: Hadoop HDFS Issue Type: New Feature Affects Versions: 0.23.1 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Fix For: 0.23.3 Attachments: HDFS-2833.patch Add to a convenience operation GETMERGE to httpfs. This will simplify for external system accessing over HTTP to consume the output of an MR job a single stream. It would have the same semantics as the 'hadoop fs -getmerge' command. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2962) jsvc needs to be removed from hdfs packaging
[
https://issues.apache.org/jira/browse/HDFS-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated HDFS-2962:
Fix Version/s: (was: 0.23.1)
0.23.3
jsvc needs to be removed from hdfs packaging
Key: HDFS-2962
URL: https://issues.apache.org/jira/browse/HDFS-2962
Project: Hadoop HDFS
Issue Type: Bug
Components: data-node, security
Affects Versions: 0.23.0
Reporter: Mingjie Lai
Fix For: 0.23.3
As we discussed at HDFS-2303:
... hit the same problem today when I started to try secured DN of 0.23. The
shipped jsvc (commons-daemon-1.0.3) with 0.23 doesn't work on my local ubuntu
11.10 (as described by atm, with libcap.2 instead of 1), nor our cluster
centos 5 (without any libcap).
The problem doesn't happen to 0.20.205 (what we use now) since
commons-daemon-1.0.2 jsvc binary doesn't link to any libcap
(http://archive.apache.org/dist/commons/daemon/binaries/1.0.2/linux/commons-daemon-1.0.2-bin-linux-${os-arch}.tar.gz)
So the current way - downloading jsvc binary - won't work for 0.23+ secured
DN.
...
jsvc shouldn't be bundled with hadoop. Instead, it should be installed by
user.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2761) Improve Hadoop subcomponent integration in Hadoop 0.23
[ https://issues.apache.org/jira/browse/HDFS-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2761: Fix Version/s: (was: 0.23.1) 0.23.3 Improve Hadoop subcomponent integration in Hadoop 0.23 -- Key: HDFS-2761 URL: https://issues.apache.org/jira/browse/HDFS-2761 Project: Hadoop HDFS Issue Type: Improvement Components: build, hdfs client, scripts Affects Versions: 0.23.0 Reporter: Roman Shaposhnik Assignee: Roman Shaposhnik Fix For: 0.23.3 Please see HADOOP-7939 for a complete description and discussion. This JIRA is for patch tracking purposes only. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2316) [umbrella] WebHDFS: a complete FileSystem implementation for accessing HDFS over HTTP
[ https://issues.apache.org/jira/browse/HDFS-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2316: Fix Version/s: (was: 0.23.1) 0.23.3 [umbrella] WebHDFS: a complete FileSystem implementation for accessing HDFS over HTTP - Key: HDFS-2316 URL: https://issues.apache.org/jira/browse/HDFS-2316 Project: Hadoop HDFS Issue Type: New Feature Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Labels: critical-0.22.0 Fix For: 1.0.0, 0.23.3 Attachments: WebHdfsAPI20111020.pdf, WebHdfsAPI2003.pdf, WebHdfsAPI2011.pdf, test-webhdfs, test-webhdfs-0.20s We current have hftp for accessing HDFS over HTTP. However, hftp is a read-only FileSystem and does not provide write accesses. In HDFS-2284, we propose to have WebHDFS for providing a complete FileSystem implementation for accessing HDFS over HTTP. The is the umbrella JIRA for the tasks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3012) Exception while renewing delegation token
[ https://issues.apache.org/jira/browse/HDFS-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-3012: Resolution: Fixed Fix Version/s: 0.23.2 Target Version/s: 0.23.2 (was: 0.23.3, 0.23.2, 0.24.0) Status: Resolved (was: Patch Available) I just merged this to 0.23.2. Thanks Bobby (and Jitendra for original review/commit). Exception while renewing delegation token - Key: HDFS-3012 URL: https://issues.apache.org/jira/browse/HDFS-3012 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Robert Joseph Evans Priority: Critical Fix For: 0.23.2 Attachments: HDFS-3012.txt, HDFS-3012.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2397) Undeprecate SecondaryNameNode
[ https://issues.apache.org/jira/browse/HDFS-2397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2397: Fix Version/s: 0.23.1 Undeprecate SecondaryNameNode - Key: HDFS-2397 URL: https://issues.apache.org/jira/browse/HDFS-2397 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.22.0, 0.23.0 Reporter: Todd Lipcon Assignee: Eli Collins Fix For: 0.23.1 Attachments: hdfs-2397.txt, hdfs-2397.txt, hdfs-2397.txt, hdfs-2397.txt I would like to consider un-deprecating the SecondaryNameNode for 0.23, and amending the documentation to indicate that it is still the most trust-worthy way to run checkpoints, and while CN/BN may have some advantages, they're not battle hardened as of yet. The test coverage for the 2NN is far superior to the CheckpointNode or BackupNode, and people have a lot more production experience. Indicating that it is deprecated before we have expanded test coverage of the CN/BN won't send the right message to our users. (For comparison, look at what a mess we got into by prematurely deprecating the old MR API before the new API had feature parity and a few versions of bug fixes). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2416) distcp with a webhdfs uri on a secure cluster fails
[ https://issues.apache.org/jira/browse/HDFS-2416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2416: Fix Version/s: (was: 0.23.1) (was: 0.24.0) 0.23.0 distcp with a webhdfs uri on a secure cluster fails --- Key: HDFS-2416 URL: https://issues.apache.org/jira/browse/HDFS-2416 Project: Hadoop HDFS Issue Type: Sub-task Affects Versions: 0.20.205.0 Reporter: Arpit Gupta Assignee: Jitendra Nath Pandey Fix For: 0.23.0, 1.0.0 Attachments: HDFS-2416-branch-0.20-security.6.patch, HDFS-2416-branch-0.20-security.7.patch, HDFS-2416-branch-0.20-security.8.patch, HDFS-2416-branch-0.20-security.patch, HDFS-2416-trunk.patch, HDFS-2416-trunk.patch, HDFS-2419-branch-0.20-security.patch, HDFS-2419-branch-0.20-security.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2527) Remove the use of Range header from webhdfs
[ https://issues.apache.org/jira/browse/HDFS-2527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2527: Fix Version/s: (was: 0.23.1) (was: 0.24.0) 0.23.0 Remove the use of Range header from webhdfs --- Key: HDFS-2527 URL: https://issues.apache.org/jira/browse/HDFS-2527 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.23.0, 1.0.0 Attachments: h2527_2001b_0.20s.patch, h2527_2002.patch, h2527_2002_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2528) webhdfs rest call to a secure dn fails when a token is sent
[
https://issues.apache.org/jira/browse/HDFS-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated HDFS-2528:
Fix Version/s: (was: 0.23.1)
(was: 0.24.0)
0.23.0
webhdfs rest call to a secure dn fails when a token is sent
---
Key: HDFS-2528
URL: https://issues.apache.org/jira/browse/HDFS-2528
Project: Hadoop HDFS
Issue Type: Sub-task
Affects Versions: 0.20.205.0
Reporter: Arpit Gupta
Assignee: Tsz Wo (Nicholas), SZE
Fix For: 0.23.0, 1.0.0
Attachments: h2528_2001.patch, h2528_2001_0.20s.patch,
h2528_2001b.patch, h2528_2001b_0.20s.patch, h2528_2002.patch,
h2528_2002_0.20s.patch, h2528_2003.patch, h2528_2003_0.20s.patch,
h2528_2003_0.20s.patch
curl -L -u : --negotiate -i
http://NN:50070/webhdfs/v1/tmp/webhdfs_data/file_small_data.txt?op=OPEN;
the following exception is thrown by the datanode when the redirect happens.
{RemoteException:{exception:IOException,javaClassName:java.io.IOException,message:Call
to failed on local exception: java.io.IOException:
javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Failed to find
any Kerberos tgt)]}}
Interestingly when using ./bin/hadoop with a webhdfs path we are able to cat
or tail a file successfully.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2539) Support doAs and GETHOMEDIRECTORY in webhdfs
[ https://issues.apache.org/jira/browse/HDFS-2539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2539: Fix Version/s: (was: 0.23.1) (was: 0.24.0) 0.23.0 Support doAs and GETHOMEDIRECTORY in webhdfs Key: HDFS-2539 URL: https://issues.apache.org/jira/browse/HDFS-2539 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.23.0, 1.0.0 Attachments: h2539_2008.patch, h2539_2008_0.20s.patch, h2539_2008_0.20s.patch, h2539_2009.patch, h2539_2009_0.20s.patch, h2539_2009b.patch, h2539_2009b_0.20s.patch, h2539_2009c.patch, h2539_2009c_0.20s.patch, h2539_2010.patch, h2539_2010_0.20s.patch, h2539_2010b.patch, h2539_2010b_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2540) Change WebHdfsFileSystem to two-step create/append
[ https://issues.apache.org/jira/browse/HDFS-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2540: Fix Version/s: (was: 0.23.1) (was: 0.24.0) 0.23.0 Change WebHdfsFileSystem to two-step create/append -- Key: HDFS-2540 URL: https://issues.apache.org/jira/browse/HDFS-2540 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.23.0, 1.0.0 Attachments: h2540_2007.patch, h2540_2007_0.20s.patch, h2540_2008.patch, h2540_2008_0.20s.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2596) TestDirectoryScanner doesn't test parallel scans
[
https://issues.apache.org/jira/browse/HDFS-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated HDFS-2596:
Fix Version/s: 0.23.1
TestDirectoryScanner doesn't test parallel scans
Key: HDFS-2596
URL: https://issues.apache.org/jira/browse/HDFS-2596
Project: Hadoop HDFS
Issue Type: Bug
Components: data-node, test
Affects Versions: 0.22.0, 0.23.0
Reporter: Eli Collins
Assignee: Eli Collins
Fix For: 0.23.1
Attachments: hdfs-2596-1.patch
The code from HDFS-854 below doesn't run the test with parallel scanning.
They probably intended parallelism 3.
{code}
+ public void testDirectoryScanner() throws Exception {
+// Run the test with and without parallel scanning
+for (int parallelism = 1; parallelism 2; parallelism++) {
+ runTest(parallelism);
+}
+ }
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2676) Remove Avro RPC
[ https://issues.apache.org/jira/browse/HDFS-2676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2676: Fix Version/s: (was: 0.23.1) Remove Avro RPC --- Key: HDFS-2676 URL: https://issues.apache.org/jira/browse/HDFS-2676 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.23.1 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 Attachments: HDFS-2676.txt, HDFS-2676.txt, HDFS-2676.txt Please see the discussion in HDFS-2660 for more details. I have created a branch HADOOP-6659 to save the Avro work, if in the future some one wants to use the work that existed to add support for Avro RPC. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2707) HttpFS should read the hadoop-auth secret from a file instead inline from the configuration
[ https://issues.apache.org/jira/browse/HDFS-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2707: Fix Version/s: (was: 0.24.0) 0.23.1 HttpFS should read the hadoop-auth secret from a file instead inline from the configuration --- Key: HDFS-2707 URL: https://issues.apache.org/jira/browse/HDFS-2707 Project: Hadoop HDFS Issue Type: Bug Components: security Affects Versions: 0.24.0, 0.23.1 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Fix For: 0.23.1 Attachments: HDFS-2707.patch, HDFS-2707.patch Similar to HADOOP-7621, the secret should be in a file other than the configuration file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2886) CreateEditLogs should generate a realistic edit log.
[
https://issues.apache.org/jira/browse/HDFS-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated HDFS-2886:
Target Version/s: 0.24.0, 0.23.1, 0.22.1 (was: 0.22.1, 0.23.1, 0.24.0)
Fix Version/s: (was: 0.23.1)
(was: 0.24.0)
CreateEditLogs should generate a realistic edit log.
Key: HDFS-2886
URL: https://issues.apache.org/jira/browse/HDFS-2886
Project: Hadoop HDFS
Issue Type: Sub-task
Components: name-node
Affects Versions: 0.24.0, 0.23.1, 0.22.1
Reporter: Konstantin Shvachko
Assignee: Konstantin Shvachko
Fix For: 0.22.1
Attachments: createLog-0.22.patch, createLog-trunk.patch
CreateEditsLog generates non-standard transactions. In real life first
transaction that creates a file does not contain blocks. While CreateEditsLog
adds blocks to this transaction. Change CreateEditsLog to produce real-life
transaction.
Also cleanup unused parameters for {{FSDirectory.updateFile()}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2893) The start/stop scripts don't start/stop the 2NN when using the default configuration
[ https://issues.apache.org/jira/browse/HDFS-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2893: Fix Version/s: 0.23.1 The start/stop scripts don't start/stop the 2NN when using the default configuration Key: HDFS-2893 URL: https://issues.apache.org/jira/browse/HDFS-2893 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.23.1 Reporter: Eli Collins Assignee: Eli Collins Priority: Minor Fix For: 0.23.1 Attachments: hdfs-2893.txt HDFS-1703 changed the behavior of the start/stop scripts so that the masters file is no longer used to indicate which hosts to start the 2NN on. The 2NN is now started, when using start-dfs.sh, on hosts only when dfs.namenode.secondary.http-address is configured with a non-wildcard IP. This means you can not start a NN using an http-address specified using a wildcard IP. We should allow a 2NN to be started with the default config, ie start-dfs.sh should start a NN, 2NN and DN. The packaging already works this way (it doesn't use start-dfs.sh, it uses hadoop-daemon.sh directly w/o first checking getconf) so let's bring start-dfs.sh in line with this behavior. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-233) Support for snapshots
[ https://issues.apache.org/jira/browse/HDFS-233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-233: --- Comment: was deleted (was: I will return to the office on January 17th. For urgent matters, please contact Aparna or Philip L. The week between Christmas and New Years', your best bet is Chris L. Thanks, -- Philip ) Support for snapshots - Key: HDFS-233 URL: https://issues.apache.org/jira/browse/HDFS-233 Project: Hadoop HDFS Issue Type: New Feature Reporter: dhruba borthakur Assignee: dhruba borthakur Attachments: Snapshots.pdf, Snapshots.pdf Support HDFS snapshots. It should support creating snapshots without shutting down the file system. Snapshot creation should be lightweight and a typical system should be able to support a few thousands concurrent snapshots. There should be a way to surface (i.e. mount) a few of these snapshots simultaneously. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2316) webhdfs: a complete FileSystem implementation for accessing HDFS over HTTP
[ https://issues.apache.org/jira/browse/HDFS-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2316: Target Version/s: 0.23.0, 0.20.205.1, 0.22.0 (was: 0.23.0, 0.20.205.1) Labels: critical-0.22.0 (was: ) webhdfs: a complete FileSystem implementation for accessing HDFS over HTTP -- Key: HDFS-2316 URL: https://issues.apache.org/jira/browse/HDFS-2316 Project: Hadoop HDFS Issue Type: New Feature Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Labels: critical-0.22.0 Fix For: 0.20.205.1, 0.23.1 Attachments: WebHdfsAPI20111020.pdf, WebHdfsAPI2003.pdf, WebHdfsAPI2011.pdf, test-webhdfs, test-webhdfs-0.20s We current have hftp for accessing HDFS over HTTP. However, hftp is a read-only FileSystem and does not provide write accesses. In HDFS-2284, we propose to have webhdfs for providing a complete FileSystem implementation for accessing HDFS over HTTP. The is the umbrella JIRA for the tasks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2445) Incorrect exit code for hadoop-hdfs-test tests when exception thrown
[ https://issues.apache.org/jira/browse/HDFS-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2445: Resolution: Fixed Fix Version/s: 0.23.0 Target Version/s: 0.23.0 (was: 0.24.0, 0.23.0) Status: Resolved (was: Patch Available) I just committed this. Thanks Jonathan! Incorrect exit code for hadoop-hdfs-test tests when exception thrown Key: HDFS-2445 URL: https://issues.apache.org/jira/browse/HDFS-2445 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.20.205.0, 0.23.0, 0.24.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Fix For: 0.23.0 Attachments: HDFS-2445.patch Please see MAPREDUCE-3179 for a full description. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
