Re: [VOTE] Release Apache Hadoop 2.1.0-beta

I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This bug can easily be reproduced by some HBase tests. I'd like this to be considered before we make a beta release. Have spoken about this with some hdfs folks offline and I am told that it is being worked on. Thanks Devaraj

Re: [VOTE] - Release 2.0.5-beta

+1 (binding) on the proposal. 2-3 weeks doesn't sound too long a time, and we have many committers willing to be on-call to fix issues when they are discovered. On Wed, May 15, 2013 at 10:57 AM, Arun C Murthy wrote: > Folks, > > A considerable number of people have expressed confusion regarding

Re: [VOTE] Hadoop-1.0.3-rc1

nefficient :) > > Thanks, > > __Luke > > On Tue, May 15, 2012 at 12:05 PM, Devaraj Das wrote: >> +1 (binding) >> I downloaded the release, built (non-secure) with IBM's JDK [IBM J9 VM >> (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux amd64-64 jvmxa6460sr10-201

Re: [VOTE] Hadoop-1.0.3-rc1

+1 (binding) I downloaded the release, built (non-secure) with IBM's JDK [IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux amd64-64 jvmxa6460sr10-20111207_96808 (JIT enabled, AOT enabled)]. Ran the sleep job successfully on a single node cluster. On May 8, 2012, at 6:25 PM, Matt Foley wrote:

Re: [VOTE] Hadoop-1.0.2-rc2

+1 Downloaded the tar ball hadoop-1.0.2.tar.gz. Built it, and then successfully ran some simple jobs on my laptop. On Mar 27, 2012, at 11:52 AM, Matt Foley wrote: > Aaron, > Under current rules, I think we have to restart the vote. Later this > afternoon I'll post a proposal for a change in th

Re: Question about Hadoop-8192 and rackToBlocks ordering

Architect and Lead > IBM Senior Software Engineer > Phone# 512-286-8393 > Fax# 512-838-8858 > > > > > > From: Devaraj Das > To: common-dev@hadoop.apache.org > Cc: Jeffrey J Heroux/Poughkeepsie/IBM@IBMUS, John >Williams/Austin/IBM@IBMUS &

Re: Question about Hadoop-8192 and rackToBlocks ordering

; > Hope this clarifies things a bit. > > Regards, > Kumar > > > Kumar Ravi > IBM Linux Technology Center > Austin, TX > > Tel.: (512)286-8179 > > Devaraj Das ---03/22/2012 04:41:36 PM---On Mar 22, 2012, at 11:45 AM, Amir > Sanjar wrote: > >

Re: Question about Hadoop-8192 and rackToBlocks ordering

On Mar 22, 2012, at 11:45 AM, Amir Sanjar wrote: > Thanks for the reply Robert, > However I believe the main design issue is: > If there is a rack ( listed in rackToBlock hashMap) that contains all the > blocks (stored in blockToNode hashMap), regardless of the order, the split > operation termin

Re: Hadoop-8192 and rackToBlocks ordering

Hi Kumar, I assume you are referring to HADOOP-8192. I need to look at the code to see if the ordering matters but most likely it wouldn't. Thanks Devaraj On Mar 21, 2012, at 1:47 PM, Kumar Ravi wrote: > > > Hello, > > We have been looking at IBM JDK junit failures on Hadoop-1.0.1 > independe

[jira] [Resolved] (HADOOP-8188) Fix the build process to do with jsvc, with IBM's JDK as the underlying jdk

[ https://issues.apache.org/jira/browse/HADOOP-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das resolved HADOOP-8188. - Resolution: Fixed Committed the patch. > Fix the build process to do with j

[jira] [Created] (HADOOP-8192) Fix unit test failures with IBM's JDK

0" Java(TM) SE Runtime Environment (build pxi3260sr10-20111208_01(SR10)) IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32 jvmxi3260sr10-20111207_96808 (JIT enabled, AOT enabled) J9VM - 20111207_096808 JIT - r9_2007_21307ifx1 GC - 20110519_AA) JCL - 2004_02 Report

[jira] [Created] (HADOOP-8188) Fix the build process to do with jsvc, with IBM's JDK as the underlying jdk

ommon Issue Type: Improvement Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 1.0.3 When IBM JDK is used as the underlying JDK for the build process, the build of jsvc fails. I just needed to add an extra "os arch" expression in the condi

[jira] [Created] (HADOOP-8187) Improve the discovery of the jvm library during the build process

: Improvement Reporter: Devaraj Das Improve the discovery of the jvm library during the build of native libraries/libhdfs/fuse-dfs, etc. A couple of different ways are currently used (discussed in HADOOP-6924). We should clean this part up and also consider builds of native stuff on

Re: [VOTE] Hadoop-1.0.1 Release Candidate 1

+1 on making a 1.0.1 release now and following up with a 1.0.2 On Feb 19, 2012, at 11:52 PM, Matt Foley wrote: > Hi, > Giri found an issue with the naming of the 64-bit RPM and DEB; see > HADOOP-8090 . The RPM > is called "hadoop-1.0.1-1.amd64.r

Re: [VOTE] Hadoop-1.0.1 Release Candidate 1

+1 Downloaded the tar ball and stood up a single-node cluster. Ran a sleep job. On Feb 14, 2012, at 2:29 PM, Matt Foley wrote: > Hi all, > Hadoop-1.0.1-rc1 is now available at >http://people.apache.org/~mattf/hadoop-1.0.1-rc1/ > Please download it, test it, and vote on whether to accept it

[jira] [Created] (HADOOP-8084) Protobuf RPC engine can be optimized to not do copying for the RPC request/response

: Hadoop Common Issue Type: Improvement Reporter: Devaraj Das Assignee: Devaraj Das -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure

[jira] [Created] (HADOOP-7987) Support setting the run-as user in unsecure mode

: security Affects Versions: 1.0.0 Reporter: Devaraj Das Fix For: 1.1.0 Some applications need to be able to perform actions (such as launch MR jobs) from map or reduce tasks. In earlier unsecure versions of hadoop (20.x), it was possible to do this by setting user.name

[jira] [Resolved] (HADOOP-6373) adding delegation token implementation

[ https://issues.apache.org/jira/browse/HADOOP-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das resolved HADOOP-6373. - Resolution: Duplicate Fix Version/s: (was: 0.23.0) Eli, the delegation token feature

[jira] [Resolved] (HADOOP-4343) Adding user and service-to-service authentication to Hadoop

[ https://issues.apache.org/jira/browse/HADOOP-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das resolved HADOOP-4343. - Resolution: Fixed Fix Version/s: 0.20.203.0 Eli, the authentication feature is there in

Re: "java.lang.Throwable: Child Error " And " Task process exit with nonzero status of 1."

Moving this to mapreduce-user (this is the right list).. Could you please look at the TaskTracker logs around the time when you see the task failure. That might have something more useful for debugging.. On Jul 10, 2011, at 8:14 PM, Michael Hu wrote: > Hi,all, >The hadoop is set up. Whenev

[jira] [Created] (HADOOP-7268) FileContext.getLocalFSFileContext() behavior needs to be fixed w.r.t tokens

Issue Type: Bug Components: fs, security Affects Versions: 0.23.0 Reporter: Devaraj Das Fix For: 0.23.0 FileContext.getLocalFSFileContext() instantiates a FileContext object upon the first call to it, and for all subsequent calls returns back that

Re: [VOTE] Release candidate 0.20.203.0-rc0

+1 based on some single node tests I did (with security ON). On 4/29/11 4:09 PM, "Owen O'Malley" wrote: I think everything is ready to go on the 0.20.203.0 release. It includes security and a lot of improvements in the capacity scheduler and JobTracker. Should we release http://people.apache.

[jira] Resolved: (HADOOP-6653) NullPointerException in setupSaslConnection when browsing directories

[ https://issues.apache.org/jira/browse/HADOOP-6653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das resolved HADOOP-6653. - Resolution: Invalid This is not applicable in trunk anymore. > NullPointerException

[jira] Created: (HADOOP-6843) Entries in the FileSystem's Cache could be cleared when they are not used

ommon Issue Type: Bug Reporter: Devaraj Das Fix For: 0.22.0 In FileSystem, there is a cache maintained for Filesystem instances. The entries in the cache are cleared only when explicit FileSystem.close is invoked. Applications are not careful on this aspect. Typically,

[jira] Created: (HADOOP-6818) Provide a JNI-based implementation of GroupMappingServiceProvider

: Improvement Components: security Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.22.0 The default implementation of GroupMappingServiceProvider does a fork of a unix command to get the groups of a user. Since the group resolution happens in the

[jira] Created: (HADOOP-6813) Add a new newInstance method in FileSystem that takes a "user" as argument

oop Common Issue Type: Improvement Components: fs Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.22.0 In order to implement HDFS-1000 for trunk, I need to have a newInstance API in FileSystem that takes a "user" as an argument.

[jira] Created: (HADOOP-6776) UserGroupInformation.createProxyUser's javadoc is broken

: Bug Components: security Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.22.0 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

[jira] Created: (HADOOP-6769) Add an API in FileSystem to get FileSystem instances based on users

Type: Improvement Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.22.0 HDFS-1000 requires an API in FileSystem to get FileSystem instances based on users. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment

[jira] Created: (HADOOP-6718) Client does not close connection when an exception happens during SASL negotiation

: Hadoop Common Issue Type: Bug Components: ipc Affects Versions: 0.22.0 Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.22.0 setupSaslConnection in the RPC client might fail to successfully set up a sasl connection (e.g. if the

[jira] Created: (HADOOP-6706) Relogin behavior for RPC clients could be improved

: security Affects Versions: 0.22.0 Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.22.0 Currently, the relogin in the RPC client happens on only a SaslException. But we have seen cases where other exceptions are thrown (like IllegalStateException when

[jira] Created: (HADOOP-6652) ShellBasedUnixGroupsMapping shouldn't have a cache

orter: Devaraj Das Assignee: Devaraj Das Fix For: 0.22.0 Attachments: groups.patch Since the Groups class already has a time based cache, the cache from ShellBasedUnixGroupsMapping should be removed. -- This message is automatically generated by JIRA. - You can rep

[jira] Created: (HADOOP-6583) Capture metrics for authentication/authorization at the RPC layer

: Improvement Components: ipc, security Affects Versions: 0.22.0 Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.22.0 Define metrics for authentication/authorization for the RPC layer. -- This message is automatically generated by JIRA

[jira] Created: (HADOOP-6582) Token class should have a toString, equals and hashcode method

: Improvement Components: security Affects Versions: 0.22.0 Reporter: Devaraj Das Assignee: Boris Shkolnik Fix For: 0.22.0 The Token.toString would be helpful in logging. The equals/hashcode would be useful in UserGroupInformation.equals (currently the

[jira] Created: (HADOOP-6567) The error messages for authentication and authorization failures can be improved

Common Issue Type: Improvement Components: security Affects Versions: 0.22.0 Reporter: Devaraj Das Fix For: 0.22.0 The error messages in case of authentication and authorization failures can be improved and made more structured. -- This message

[jira] Created: (HADOOP-6566) Hadoop daemons should not start up if the ownership/permissions on the directories used at runtime are misconfigured

://issues.apache.org/jira/browse/HADOOP-6566 Project: Hadoop Common Issue Type: New Feature Components: security Reporter: Devaraj Das Fix For: 0.22.0 The Hadoop daemons (like datanode, namenode) should refuse to start up if the ownership

[jira] Created: (HADOOP-6559) The RPC client should try to re-login when it detects that the TGT expired

Issue Type: Improvement Components: security Affects Versions: 0.22.0 Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.22.0 Currently while making RPC calls, the client will throw an exception if the client is not able to use the TGT

[jira] Created: (HADOOP-6552) KEYTAB_KERBEROS_OPTIONS in UserGroupInformation should have options for automatic renewal of keytab based tickets

://issues.apache.org/jira/browse/HADOOP-6552 Project: Hadoop Common Issue Type: Bug Components: security Affects Versions: 0.22.0 Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.22.0 KEYTAB_KERBEROS_OPTIONS in

[jira] Created: (HADOOP-6547) Move the Delegation Token feature to common since both HDFS and MapReduce needs it

: Hadoop Common Issue Type: New Feature Components: security Affects Versions: 0.22.0 Reporter: Devaraj Das Fix For: 0.22.0 Move the Delegation Token feature to common since both HDFS and MapReduce needs it. -- This message is automatically generated

[jira] Created: (HADOOP-6545) Cached FileSystem objects can lead to wrong token being used in setting up connections

Project: Hadoop Common Issue Type: Bug Components: security Affects Versions: 0.22.0 Reporter: Devaraj Das Fix For: 0.22.0 The FileSystem class caches the filesystem objects that it creates for users. For some cases, e.g., if the filesystem object is

[jira] Created: (HADOOP-6538) Set hadoop.security.authentication to "simple" by default

Type: Bug Components: security Affects Versions: 0.22.0 Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.22.0 The default value of "hadoop.security.authentication" is "kerberos". It makes sense for that to be "simple

[jira] Reopened: (HADOOP-6325) need security keys storage solution

[ https://issues.apache.org/jira/browse/HADOOP-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das reopened HADOOP-6325: - I think we should leave this issue open. We will need a solution to support the use case of

[jira] Resolved: (HADOOP-6339) SequenceFile writer does not properly flush stream with external DataOutputStream

[ https://issues.apache.org/jira/browse/HADOOP-6339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das resolved HADOOP-6339. - Resolution: Invalid Hey Jonathan, this is supposed to work that way. The outputstream is

Re: [VOTE] port HADOOP-6218 (Split TFile by Record Sequence Number) to hadoop 0.20/0.21

After an offline discussion with Hong and others on this subject, it seems to make sense. +1 On 10/12/09 3:55 PM, "Hong Tang" wrote: HADOOP-6218 exposed the internal "Location" object as a global Record Sequence Number (RecNum). The feature is useful in a number of ways: (1) support progress r

Re: [VOTE] Should we release Common 0.20.1-rc0?

I just committed MAPREDUCE-924. This probably needs to be there in 0.20.1. Also, the fix for HADOOP-6227 should probably be there in 0.20.1. We hit the bug in our production cluster recently. Thoughts? On 8/29/09 5:36 AM, "Owen O'Malley" wrote: I've rolled a release candidate for 0.20.1. Plea

Re: [VOTE] Freeze date for Common, HDFS, and MapReduce 0.21

+1 On 8/13/09 4:34 AM, "Owen O'Malley" wrote: All, After the discussion settled last time, it seems that HDFS needs more time to settle append and sync. Therefore, I'd like to propose a freeze time of 4:30 pst on 18 Sep for making the 0.21 branch for Common, HDFS, and MapReduce. -- Owen

Re: [VOTE] Back-port TFile to Hadoop 0.20

+1 On 7/8/09 12:25 AM, "Hong Tang" wrote: I have talked with a few folks in the community who are interested in using TFile (HADOOP-3315) in their projects that are currently dependent on Hadoop 0.20, and it would significantly simplify the release process as well as their lives if we could bac