[jira] [Created] (HADOOP-8787) KerberosAuthenticationHandler should include missing property names in configuration

[Todd Lipcon (JIRA)]

Todd Lipcon created HADOOP-8787:
---

 Summary: KerberosAuthenticationHandler should include missing 
property names in configuration
 Key: HADOOP-8787
 URL: https://issues.apache.org/jira/browse/HADOOP-8787
 Project: Hadoop Common
  Issue Type: Improvement
  Components: security
Affects Versions: 2.0.1-alpha, 1.0.3, 3.0.0
Reporter: Todd Lipcon
Priority: Minor


Currently, if the spnego keytab is missing from the configuration, the user 
gets an error like: "javax.servlet.ServletException: Principal not defined in 
configuration". This should be augmented to actually show the configuration 
variable which is missing. Otherwise it is hard for a user to know what to fix.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8786) HttpServer continues to start even if AuthenticationFilter fails to init

[Todd Lipcon (JIRA)]

Todd Lipcon created HADOOP-8786:
---

 Summary: HttpServer continues to start even if 
AuthenticationFilter fails to init
 Key: HADOOP-8786
 URL: https://issues.apache.org/jira/browse/HADOOP-8786
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 2.0.1-alpha, 1.2.0, 3.0.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon


As seen in HDFS-3904, if the AuthenticationFilter fails to initialize, the web 
server will continue to start up. We need to check for context initialization 
errors after starting the server.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8785) Remove security conditionals from FileSystems

[Daryn Sharp (JIRA)]

Daryn Sharp created HADOOP-8785:
---

 Summary: Remove security conditionals from FileSystems
 Key: HADOOP-8785
 URL: https://issues.apache.org/jira/browse/HADOOP-8785
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs, security
Affects Versions: 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp


Streamline {{FileSystem}} classes by removing {{UGI.isSecurityEnabled()}} 
conditionals.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8784) Improve IPC.Client's token use

[Daryn Sharp (JIRA)]

Daryn Sharp created HADOOP-8784:
---

 Summary: Improve IPC.Client's token use
 Key: HADOOP-8784
 URL: https://issues.apache.org/jira/browse/HADOOP-8784
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: ipc, security
Affects Versions: 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp


If present, tokens should be sent for all auth types including simple auth.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8783) Improve RPC.Server

[Daryn Sharp (JIRA)]

Daryn Sharp created HADOOP-8783:
---

 Summary: Improve RPC.Server
 Key: HADOOP-8783
 URL: https://issues.apache.org/jira/browse/HADOOP-8783
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: ipc, security
Affects Versions: 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8782) Make serialization of Delegation Tokens extensible

[Owen O'Malley (JIRA)]

Owen O'Malley created HADOOP-8782:
-

 Summary: Make serialization of Delegation Tokens extensible
 Key: HADOOP-8782
 URL: https://issues.apache.org/jira/browse/HADOOP-8782
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Owen O'Malley
Assignee: Owen O'Malley


Delegation tokens use of Writables makes it difficult to extend them with new 
fields. Moving to protobufs as the scheme will make it much more extensible.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8781) hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH

[Alejandro Abdelnur (JIRA)]

Alejandro Abdelnur created HADOOP-8781:
--

 Summary: hadoop-config.sh should add JAVA_LIBRARY_PATH to 
LD_LIBRARY_PATH
 Key: HADOOP-8781
 URL: https://issues.apache.org/jira/browse/HADOOP-8781
 Project: Hadoop Common
  Issue Type: Bug
  Components: scripts
Affects Versions: 2.0.2-alpha
Reporter: Alejandro Abdelnur
 Fix For: 2.0.2-alpha


Snappy SO fails to load properly if LD_LIBRARY_PATH does not include the path 
where snappy SO is. This is observed in setups that don't have an independent 
snappy installation (not installed by Hadoop)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [DISCUSS] release branching scheme under maven was [VOTE] 0.23.3 release

[Robert Evans]

I forked the thread, because it is not really about the release vote any
more, although we both seem to be on the same page so this may be overkill
:)

On 9/10/12 3:50 PM, "Owen O'Malley"  wrote:

>On Mon, Sep 10, 2012 at 12:19 PM, Robert Evans 
>wrote:
>> Thanks for the info Owen I was not aware of that, I can see at the
>> beginning of the twiki
>> http://wiki.apache.org/hadoop/HowToReleasePostMavenization that it is
>>kind
>> of implied by the skip this section comment.  But, I was just confused
>> because to do an official release I need to change the version number in
>> several different places, and I didn¹t think that is was all that kosher
>> to do that directly in the tags area.
>
>Please don't make edits in the tags area.

Totally concur on that.

>
>>  It also seems to be a bit different
>> then what is happening on branch-2.
>
>Branch 2 also has issues, which we've also been ironing out. The
>branch-2.0.1-alpha and branch-2.0.2-alpha will be deleted soon.

Sounds good.

>
>> I was planning on deleting the branch once the vote for 0.23.3 finished,
>
>That would be fine.
>
>> but I can delete it now if you would prefer and the roll back the
>>version
>> numbers on branch-0.23 to be 0.23.3? Or would it be 0.23.3-SNAPSHOT
>>still?
>
>You can wait for 0.23.3 to clean up. The only odd part of that is that
>the 0.23.3 tag won't be on the branch-0.23.

Yes that is ugly.  I agree.

>
>With Ant, we could just override the version on the command line.
>Clearly Maven is a little pickier. You could:
>1. Set the version to 0.23.3
>2. Check in
>3. Make the tag
>4. Set the version to 0.23.4-SNAPSHOT
>5. Check in
>
>I'd also be ok with just waiting with 4+5 until the vote passes.
>Thoughts? (Of course as discussed this would be more about 0.23.4 :) )

I am fine with either way.  If we do go all they way to 5 and there are
blockers that force a re-spin of the RC then we are back to creating a new
branch off of the original tag to make changes and we get a tag that is
not off of branch-X.Y. But, if we don't go all the way to 5 then the
branch must stay locked for a week, which is not the end of the world, but
in some cases will slow down development a little.  It is a balancing
game, but I don't see major drawbacks to either situation.

>
>>  I also want to be sure that we want to not allow anyone to check
>>anything
>> in for a week+ on branch-0.23.  There are a few things I know of that
>>are
>> almost ready but are not Blockers.
>
>Sure. That makes sense.
>
>-- Owen

--Bobby

Re: [VOTE] 0.23.3 release

[Owen O'Malley]

On Mon, Sep 10, 2012 at 12:19 PM, Robert Evans  wrote:
> Thanks for the info Owen I was not aware of that, I can see at the
> beginning of the twiki
> http://wiki.apache.org/hadoop/HowToReleasePostMavenization that it is kind
> of implied by the skip this section comment.  But, I was just confused
> because to do an official release I need to change the version number in
> several different places, and I didn¹t think that is was all that kosher
> to do that directly in the tags area.

Please don't make edits in the tags area.

>  It also seems to be a bit different
> then what is happening on branch-2.

Branch 2 also has issues, which we've also been ironing out. The
branch-2.0.1-alpha and branch-2.0.2-alpha will be deleted soon.

> I was planning on deleting the branch once the vote for 0.23.3 finished,

That would be fine.

> but I can delete it now if you would prefer and the roll back the version
> numbers on branch-0.23 to be 0.23.3? Or would it be 0.23.3-SNAPSHOT still?

You can wait for 0.23.3 to clean up. The only odd part of that is that
the 0.23.3 tag won't be on the branch-0.23.

With Ant, we could just override the version on the command line.
Clearly Maven is a little pickier. You could:
1. Set the version to 0.23.3
2. Check in
3. Make the tag
4. Set the version to 0.23.4-SNAPSHOT
5. Check in

I'd also be ok with just waiting with 4+5 until the vote passes.
Thoughts? (Of course as discussed this would be more about 0.23.4 :) )

>  I also want to be sure that we want to not allow anyone to check anything
> in for a week+ on branch-0.23.  There are a few things I know of that are
> almost ready but are not Blockers.

Sure. That makes sense.

-- Owen

Re: [VOTE] 0.23.3 release

[Robert Evans]

Thanks for the info Owen I was not aware of that, I can see at the
beginning of the twiki
http://wiki.apache.org/hadoop/HowToReleasePostMavenization that it is kind
of implied by the skip this section comment.  But, I was just confused
because to do an official release I need to change the version number in
several different places, and I didn¹t think that is was all that kosher
to do that directly in the tags area.  It also seems to be a bit different
then what is happening on branch-2.

http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/

http://svn.apache.org/viewvc/hadoop/common/branches/branch-2.0.1-alpha/

http://svn.apache.org/viewvc/hadoop/common/branches/branch-2.0.2-alpha/

I was planning on deleting the branch once the vote for 0.23.3 finished,
but I can delete it now if you would prefer and the roll back the version
numbers on branch-0.23 to be 0.23.3? Or would it be 0.23.3-SNAPSHOT still?
 I also want to be sure that we want to not allow anyone to check anything
in for a week+ on branch-0.23.  There are a few things I know of that are
almost ready but are not Blockers.

--Bobby 

On 9/10/12 12:33 PM, "Owen O'Malley"  wrote:

>On Fri, Sep 7, 2012 at 12:56 PM, Bobby Evans  wrote:
>> I have built an RC0 for 0.23.3 and am calling a vote on it
>
>Hi Bobby,
>   I noticed that you created a branch-0.23.3. The pattern we have
>used in Hadoop is that there is a release branch that is for a minor
>release (branch-0.23) and tags for the release candidates
>(release-0.23.3-rc0) and releases (release-0.23.3). Generally, the RM
>freezes the minor branch while they are trying to get the right set of
>patches together for the release candidate.
>
>-- Owen

[jira] [Resolved] (HADOOP-8752) Update website to reflect merged committer lists

[Doug Cutting (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-8752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Cutting resolved HADOOP-8752.
--

   Resolution: Duplicate
Fix Version/s: site
 Assignee: (was: Eli Collins)

This was subsumed by HADOOP-8662.

> Update website to reflect merged committer lists
> 
>
> Key: HADOOP-8752
> URL: https://issues.apache.org/jira/browse/HADOOP-8752
> Project: Hadoop Common
>  Issue Type: Task
>Reporter: Eli Collins
>Priority: Minor
> Fix For: site
>
>
> Let's update the website to reflect the merged committer list 
> (http://s.apache.org/Owx), ie one top-level credits section next to the PMC 
> section. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HADOOP-8662) remove separate pages for Common, HDFS & MR projects

[Doug Cutting (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Cutting resolved HADOOP-8662.
--

Resolution: Fixed

I committed this.  The new site is now live.

Konstantin: I removed your affiliation from the who.html page.
Chris: I updated the affiliations you listed.

> remove separate pages for Common, HDFS & MR projects
> 
>
> Key: HADOOP-8662
> URL: https://issues.apache.org/jira/browse/HADOOP-8662
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: site
>Reporter: Doug Cutting
>Assignee: Doug Cutting
>Priority: Minor
> Fix For: site
>
> Attachments: HADOOP-8662.patch, HADOOP-8662.patch
>
>
> The tabs on the top of http://hadoop.apache.org/ link to separate sites for 
> Common, HDFS and MapReduce modules.  These sites are identical except for the 
> mailing lists.  I propose we move the mailing list information to the TLP 
> mailing list page and remove these sub-project websites.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [VOTE] 0.23.3 release

[Owen O'Malley]

On Fri, Sep 7, 2012 at 12:56 PM, Bobby Evans  wrote:
> I have built an RC0 for 0.23.3 and am calling a vote on it

Hi Bobby,
   I noticed that you created a branch-0.23.3. The pattern we have
used in Hadoop is that there is a release branch that is for a minor
release (branch-0.23) and tags for the release candidates
(release-0.23.3-rc0) and releases (release-0.23.3). Generally, the RM
freezes the minor branch while they are trying to get the right set of
patches together for the release candidate.

-- Owen

YARN jobs takes too long to start

[Radim Kolar]

I noticed that after some update on branch 0.23 yarn jobs are about 4 
minutes in PREP stage. I tried to track source of problem at it seems 
that chmod 0700 is called on every class file unpacked from job JAR.


Can you point me to code which is doing it, hadoop also supports 
recursive chmod which should be used instead of fork/exec chmod

Build failed in Jenkins: Hadoop-Common-trunk #529

[Apache Jenkins Server]

See 

--
[...truncated 26940 lines...]
[DEBUG]   (s) debug = false
[DEBUG]   (s) effort = Default
[DEBUG]   (s) failOnError = true
[DEBUG]   (s) findbugsXmlOutput = false
[DEBUG]   (s) findbugsXmlOutputDirectory = 

[DEBUG]   (s) fork = true
[DEBUG]   (s) includeTests = false
[DEBUG]   (s) localRepository =id: local
  url: file:///home/jenkins/.m2/repository/
   layout: none

[DEBUG]   (s) maxHeap = 512
[DEBUG]   (s) nested = false
[DEBUG]   (s) outputDirectory = 

[DEBUG]   (s) outputEncoding = UTF-8
[DEBUG]   (s) pluginArtifacts = 
[org.codehaus.mojo:findbugs-maven-plugin:maven-plugin:2.3.2:, 
com.google.code.findbugs:bcel:jar:1.3.9:compile, 
org.codehaus.gmaven:gmaven-mojo:jar:1.3:compile, 
org.codehaus.gmaven.runtime:gmaven-runtime-api:jar:1.3:compile, 
org.codehaus.gmaven.feature:gmaven-feature-api:jar:1.3:compile, 
org.codehaus.gmaven.runtime:gmaven-runtime-1.5:jar:1.3:compile, 
org.codehaus.gmaven.feature:gmaven-feature-support:jar:1.3:compile, 
org.codehaus.groovy:groovy-all-minimal:jar:1.5.8:compile, 
org.apache.ant:ant:jar:1.7.1:compile, 
org.apache.ant:ant-launcher:jar:1.7.1:compile, jline:jline:jar:0.9.94:compile, 
org.codehaus.plexus:plexus-interpolation:jar:1.1:compile, 
org.codehaus.gmaven:gmaven-plugin:jar:1.3:compile, 
org.codehaus.gmaven.runtime:gmaven-runtime-loader:jar:1.3:compile, 
org.codehaus.gmaven.runtime:gmaven-runtime-support:jar:1.3:compile, 
org.sonatype.gshell:gshell-io:jar:2.0:compile, 
com.thoughtworks.qdox:qdox:jar:1.10:compile, 
org.apache.maven.shared:file-management:jar:1.2.1:compile, 
org.apache.maven.shared:maven-shared-io:jar:1.1:compile, 
commons-lang:commons-lang:jar:2.4:compile, 
org.slf4j:slf4j-api:jar:1.5.10:compile, 
org.sonatype.gossip:gossip:jar:1.2:compile, 
org.apache.maven.reporting:maven-reporting-impl:jar:2.1:compile, 
commons-validator:commons-validator:jar:1.2.0:compile, 
commons-beanutils:commons-beanutils:jar:1.7.0:compile, 
commons-digester:commons-digester:jar:1.6:compile, 
commons-logging:commons-logging:jar:1.0.4:compile, oro:oro:jar:2.0.8:compile, 
xml-apis:xml-apis:jar:1.0.b2:compile, 
org.codehaus.groovy:groovy-all:jar:1.7.4:compile, 
org.apache.maven.reporting:maven-reporting-api:jar:3.0:compile, 
org.apache.maven.doxia:doxia-core:jar:1.1.3:compile, 
org.apache.maven.doxia:doxia-logging-api:jar:1.1.3:compile, 
xerces:xercesImpl:jar:2.9.1:compile, 
commons-httpclient:commons-httpclient:jar:3.1:compile, 
commons-codec:commons-codec:jar:1.2:compile, 
org.apache.maven.doxia:doxia-sink-api:jar:1.1.3:compile, 
org.apache.maven.doxia:doxia-decoration-model:jar:1.1.3:compile, 
org.apache.maven.doxia:doxia-site-renderer:jar:1.1.3:compile, 
org.apache.maven.doxia:doxia-module-xhtml:jar:1.1.3:compile, 
org.apache.maven.doxia:doxia-module-fml:jar:1.1.3:compile, 
org.codehaus.plexus:plexus-i18n:jar:1.0-beta-7:compile, 
org.codehaus.plexus:plexus-velocity:jar:1.1.7:compile, 
org.apache.velocity:velocity:jar:1.5:compile, 
commons-collections:commons-collections:jar:3.2:compile, 
org.apache.maven.shared:maven-doxia-tools:jar:1.2.1:compile, 
commons-io:commons-io:jar:1.4:compile, 
com.google.code.findbugs:findbugs-ant:jar:1.3.9:compile, 
com.google.code.findbugs:findbugs:jar:1.3.9:compile, 
com.google.code.findbugs:jsr305:jar:1.3.9:compile, 
com.google.code.findbugs:jFormatString:jar:1.3.9:compile, 
com.google.code.findbugs:annotations:jar:1.3.9:compile, 
dom4j:dom4j:jar:1.6.1:compile, jaxen:jaxen:jar:1.1.1:compile, 
jdom:jdom:jar:1.0:compile, xom:xom:jar:1.0:compile, 
xerces:xmlParserAPIs:jar:2.6.2:compile, xalan:xalan:jar:2.6.0:compile, 
com.ibm.icu:icu4j:jar:2.6.1:compile, asm:asm:jar:3.1:compile, 
asm:asm-analysis:jar:3.1:compile, asm:asm-commons:jar:3.1:compile, 
asm:asm-util:jar:3.1:compile, asm:asm-tree:jar:3.1:compile, 
asm:asm-xml:jar:3.1:compile, jgoodies:plastic:jar:1.2.0:compile, 
org.codehaus.plexus:plexus-resources:jar:1.0-alpha-4:compile, 
org.codehaus.plexus:plexus-utils:jar:1.5.1:compile]
[DEBUG]   (s) project = MavenProject: 
org.apache.hadoop:hadoop-common-project:3.0.0-SNAPSHOT @ 

[DEBUG]   (s) relaxed = false
[DEBUG]   (s) remoteArtifactRepositories = [   id: apache.snapshots.https
  url: https://repository.apache.org/content/repositories/snapshots
   layout: default
snapshots: [enabled => true, update => daily]
 releases: [enabled => true, update => daily]
,id: repository.jboss.org
  url: http://repository.jboss.org/nexus/content/groups/public/
   layout: default
snapshots: [enabled => false, update => daily]
 releases: [enabled => true, update => daily]
,id: central
  url: http://repo1.maven.org/maven2
   layout: default
snapshots:

[jira] [Created] (HADOOP-8780) Update DeprecatedProperties apt file

[Ahmed Radwan (JIRA)]

Ahmed Radwan created HADOOP-8780:


 Summary: Update DeprecatedProperties apt file
 Key: HADOOP-8780
 URL: https://issues.apache.org/jira/browse/HADOOP-8780
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Ahmed Radwan


The current list of deprecated properties is not up-to-date. I'll will upload a 
patch momentarily.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira