[jira] [Created] (HADOOP-10821) Prepare the release notes for Hadoop 2.5.0

[Akira AJISAKA (JIRA)]

Akira AJISAKA created HADOOP-10821:
--

 Summary: Prepare the release notes for Hadoop 2.5.0
 Key: HADOOP-10821
 URL: https://issues.apache.org/jira/browse/HADOOP-10821
 Project: Hadoop Common
  Issue Type: Task
Affects Versions: 2.5.0
Reporter: Akira AJISAKA
Priority: Blocker


The release notes for 2.3.0+ still talk about federation and MRv2
being new features. We should update them.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: 'current' document links to 2.3.0

[Akira AJISAKA]

Thanks Andrew!

> One thing we can do though is have proper notes for 2.5 :)

Filed https://issues.apache.org/jira/browse/HADOOP-10821

(2014/07/12 6:49), Andrew Wang wrote:

I just moved the symlink in the site svn repo according to step 12 in [1],
dunno when it'll get propagated.

The release notes for 2.3.0+ also still talk about federation and MRv2
being new features. I think it's generated from the release tarball, so we
probably should  have fixed these notes before releasing. Not sure how we'd
go about fixing this.

One thing we can do though is have proper notes for 2.5 :)

[1] http://wiki.apache.org/hadoop/HowToRelease


On Thu, Jul 10, 2014 at 11:40 PM, Akira AJISAKA 
wrote:


Hi,

I noticed http://hadoop.apache.org/docs/current/ linked to 2.3.0.
Now 2.4.1 is the latest release, would you please update the link?

Thanks,
Akira

Build failed in Jenkins: Hadoop-Common-0.23-Build #1010

[Apache Jenkins Server]

See 

--
[...truncated 15919 lines...]
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.942 sec
Running org.apache.hadoop.security.TestCredentials
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.646 sec
Running org.apache.hadoop.security.token.delegation.TestDelegationToken
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.027 sec
Running org.apache.hadoop.security.token.TestToken
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.148 sec
Running org.apache.hadoop.security.authorize.TestProxyUsers
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.493 sec
Running org.apache.hadoop.security.authorize.TestAccessControlList
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.445 sec
Running org.apache.hadoop.security.TestAuthenticationFilter
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.383 sec
Running org.apache.hadoop.security.TestJNIGroupsMapping
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.165 sec
Running org.apache.hadoop.security.TestProxyUserFromEnv
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.363 sec
Running org.apache.hadoop.security.TestUserFromEnv
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.357 sec
Running org.apache.hadoop.security.TestSecurityUtil
Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.803 sec
Running org.apache.hadoop.security.TestGroupFallback
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.425 sec
Running org.apache.hadoop.security.TestGroupsCaching
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.278 sec
Running org.apache.hadoop.util.TestStringInterner
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.117 sec
Running org.apache.hadoop.util.TestOptions
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.08 sec
Running org.apache.hadoop.util.TestRunJar
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.127 sec
Running org.apache.hadoop.util.TestAsyncDiskService
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.127 sec
Running org.apache.hadoop.util.TestDataChecksum
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.187 sec
Running org.apache.hadoop.util.TestGenericOptionsParser
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.689 sec
Running org.apache.hadoop.util.TestHostsFileReader
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.186 sec
Running org.apache.hadoop.util.TestIndexedSort
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.608 sec
Running org.apache.hadoop.util.TestGenericsUtil
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.26 sec
Running org.apache.hadoop.util.TestPureJavaCrc32
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.307 sec
Running org.apache.hadoop.util.TestStringUtils
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.138 sec
Running org.apache.hadoop.util.TestProtoUtil
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.079 sec
Running org.apache.hadoop.util.TestDiskChecker
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.487 sec
Running org.apache.hadoop.util.TestShell
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.194 sec
Running org.apache.hadoop.util.TestShutdownHookManager
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.145 sec
Running org.apache.hadoop.util.TestJarFinder
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.841 sec
Running org.apache.hadoop.util.TestReflectionUtils
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.519 sec
Running org.apache.hadoop.io.TestSequenceFile
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.947 sec
Running org.apache.hadoop.io.TestSecureIOUtils
Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.499 sec
Running org.apache.hadoop.io.nativeio.TestNativeIO
Tests run: 9, Failures: 0, Errors: 0, Skipped: 9, Time elapsed: 0.161 sec
Running org.apache.hadoop.io.TestWritable
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.071 sec
Running org.apache.hadoop.io.TestIOUtils
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.281 sec
Running org.apache.hadoop.io.TestTextNonUTF8
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.047 sec
Running org.apache.hadoop.io.TestMD5Hash
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.149 sec
Running org.apache.hadoop.io.TestMapFile
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.722 sec
Running org.apache.hadoop.io.TestWritableName
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.137 sec
Running org.apache.hadoop.io.TestSortedMapWritable
Tests run: 4, Failur

Re: Porting Hadoop to FreeBSD

[Colin McCabe]

Thanks for working on this, Dmitry.  It's good to see Hadoop support
another platform.  The code changes look pretty minor, too.

best,
Colin

On Tue, Jul 8, 2014 at 7:08 AM, Dmitry Sivachenko  wrote:
> Hello,
>
> I am trying to make hadoop usable on FreeBSD OS.  Following Steve Loughran's 
> suggestion, I created HADOOP-10796 to accumulate links to other issues 
> related to this subject.
> I added links to (almost) all patches needed to achieve this goal.  Right now 
> I have a small cluster running Hadoop-2.4.1 on FreeBSD and passing basic 
> tests (TeraSort mainly).
>
> I am looking forward for cooperation with you to include all relevant changes 
> into hadoop distribution.
> Please do not hesitate to contact me if you are not satisfied with the 
> proposed solutions.
>
> Thanks in advance!

Re: [jira] [Created] (HADOOP-10795) unale to build hadoop 2.4.1(redhat5.8 x64)

[Colin McCabe]

A unit test failed.  See my response on JIRA.

best,
Colin

On Tue, Jul 8, 2014 at 3:11 PM, Jay Vyas  wrote:
> these appear to be java errors related use to your jdk?
> maybe your JDK doesnt match up well with your OS.
> Consider trying red hat 6+ or Fedora 20?
>
>
> On Jul 8, 2014, at 5:45 AM, "moses.wang (JIRA)"  wrote:
>
>> moses.wang created HADOOP-10795:
>> ---
>>
>> Summary: unale to build hadoop 2.4.1(redhat5.8 x64)
>> Key: HADOOP-10795
>> URL: https://issues.apache.org/jira/browse/HADOOP-10795
>> Project: Hadoop Common
>>  Issue Type: Bug
>>  Components: build
>> Environment: OS version rehat 5.8 x64
>> maven version 3.3.1
>> java version jdk 1.7_15 for x64
>>Reporter: moses.wang
>>
>>
>> unale to build hadoop 2.4.1(redhat5.8 x64)
>> WARNING] Some problems were encountered while building the effective model 
>> for org.apache.hadoop:hadoop-project:pom:2.4.1
>> [WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but 
>> found duplicate declaration of plugin 
>> org.apache.maven.plugins:maven-enforcer-plugin @ line 1015, column 15
>>
>> [WARNING] Some problems were encountered while building the effective model 
>> for org.apache.hadoop:hadoop-common:jar:2.4.1
>> [WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but 
>> found duplicate declaration of plugin 
>> org.apache.maven.plugins:maven-surefire-plugin @ line 479, column 15
>> [WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but 
>> found duplicate declaration of plugin 
>> org.apache.maven.plugins:maven-enforcer-plugin @ 
>> org.apache.hadoop:hadoop-project:2.4.1, 
>> /home/software/Server/hadoop-2.4.1-src/hadoop-project/pom.xml, line 1015, 
>> column 15
>>
>> WARNING] 
>> /home/software/Server/hadoop-2.4.1-src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/FastByteComparisons.java:[25,15]
>>  Unsafe is internal proprietary API and may be removed in a future release
>> [WARNING] 
>> /home/software/Server/hadoop-2.4.1-src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java:[42,15]
>>  Unsafe is internal proprietary API and may be removed in a future release
>> [WARNING] 
>> /home/software/Server/hadoop-2.4.1-src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SignalLogger.java:[21,15]
>>  Signal is internal proprietary API and may be removed in a future release
>> [WARNING] 
>> /home/software/Server/hadoop-2.4.1-src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SignalLogger.java:[22,15]
>>  SignalHandler is internal proprietary API and may be removed in a future 
>> release
>>
>> [WARNING] 
>> /home/software/Server/hadoop-2.4.1-src/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/ssl/KeyStoreTestUtil.java:[22,24]
>>  AlgorithmId is internal proprietary API and may be removed in a future 
>> release
>> [WARNING] 
>> /home/software/Server/hadoop-2.4.1-src/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/ssl/KeyStoreTestUtil.java:[23,24]
>>  CertificateAlgorithmId is internal proprietary API and may be removed in a 
>> future release
>>
>> testROBufferDirAndRWBufferDir[1](org.apache.hadoop.fs.TestLocalDirAllocator) 
>>  Time elapsed: 0.014 sec  <<< FAILURE!
>>
>> [ERROR] Failed to execute goal 
>> org.apache.maven.plugins:maven-surefire-plugin:2.16:test (default-test) on 
>> project hadoop-common: There are test failures.
>> [ERROR]
>>
>>
>>
>> --
>> This message was sent by Atlassian JIRA
>> (v6.2#6252)
>

Phone bridge info [Re: Meetup invitation: Consensus based replication in Hadoop]

[Konstantin Boudnik]

To enable remote people to join the conversation, we'll keep open the
following GotoMeeting session. You can join either from your computer or by
using dial-in number. 

 =
Please join my meeting, Jul 15, 2014 at 12:00 PM PDT.
https://www4.gotomeeting.com/join/357791919

2. Use your microphone and speakers (VoIP) - a headset is recommended. Or,
call in using your telephone.

Dial +1 (805) 309-0014
Access Code: 357-791-919
Audio PIN: Shown after joining the meeting

Meeting ID: 357-791-919
 =

BTW, we have one or two spots left, so if you didn't make your reservation -
now is the time.

See you all tomorrow.
  Cos

On Fri, Jul 11, 2014 at 04:33PM, Konstantin Boudnik wrote:
> One more update: it seema that for ppl in SF, who oftentimes might not even
> have a car, getting to San Ramon can represent a certain difficulty.
> 
> So we'll do a shuttle pickup from West Dublin BART station if there's at least
> a few people who want to use the option.
> 
> Please respond directly to me if you're interested before end of Sunday, 13th.
> Cheers,
>   Cos
> 
> On Tue, Jul 08, 2014 at 12:23PM, Konstantin Boudnik wrote:
> > All,
> > 
> > Re-sending this announcement in case it fell through over the long weekend
> > when people were away. We still have seats left, so register soon.
> > 
> > Regards,
> >   Cos
> > 
> > On Wed, Jul 02, 2014 at 06:37PM, Konstantin Boudnik wrote:
> > > We'd like to invite you to the 
> > > Consensus based replication in Hadoop: A deep dive
> > > event that we are happy to hold in our San Ramon office on July 15th at 
> > > noon.
> > > We'd like to accommodate as many people as possible, but I think are 
> > > physically
> > > limited to 30 (+/- a few), so please RSVP to this Eventbrite invitation:
> > > 
> > > https://www.eventbrite.co.uk/e/consensus-based-replication-in-hadoop-a-deep-dive-tickets-12158236613
> > > 
> > > We'll provide pizza and beverages (feel free to express your special 
> > > dietary
> > > requirements if any).
> > > 
> > > See you soon!
> > > With regards,
> > >   Cos
> > > 
> > > On Wed, Jun 18, 2014 at 08:45PM, Konstantin Boudnik wrote:
> > > > Guys,
> > > > 
> > > > In the last a couple of weeks, we had a very good and productive 
> > > > initial round
> > > > of discussions on the JIRAs. I think it is worthy to keep the momentum 
> > > > going
> > > > and have a more detailed conversation. For that, we'd like to host s 
> > > > Hadoop
> > > > developers meetup to get into the bowls of the consensus-based 
> > > > coordination
> > > > implementation for HDFS. The proposed venue is our office in San Ramon, 
> > > > CA.
> > > > 
> > > > Considering that it is already a mid week and the following one looks 
> > > > short
> > > > because of the holidays, how would the week of July 7th looks for yall?
> > > > Tuesday or Thursday look pretty good on our end.
> > > > 
> > > > Please chime in on your preference either here or reach of directly to 
> > > > me.
> > > > Once I have a few RSVPs I will setup an event on Eventbrite or similar.
> > > > 
> > > > Looking forward to your input. Regards,
> > > >   Cos
> > > > 
> > > > On Thu, May 29, 2014 at 02:09PM, Konstantin Shvachko wrote:
> > > > > Hello hadoop developers,
> > > > > 
> > > > > I just opened two jiras proposing to introduce ConsensusNode into 
> > > > > HDFS and
> > > > > a Coordination Engine into Hadoop Common. The latter should benefit 
> > > > > HDFS
> > > > > and  HBase as well as potentially other projects. See HDFS-6469 and
> > > > > HADOOP-10641 for details.
> > > > > The effort is based on the system we built at Wandisco with my 
> > > > > colleagues,
> > > > > who are glad to contribute it to Apache, as quite a few people in the
> > > > > community expressed interest in this ideas and their potential 
> > > > > applications.
> > > > > 
> > > > > We should probably keep technical discussions in the jiras. Here on 
> > > > > the dev
> > > > > list I wanted to touch-base on any logistic issues / questions.
> > > > > - First of all, any ideas and help are very much welcome.
> > > > > - We would like to set up a meetup to discuss this if people are
> > > > > interested. Hadoop Summit next week may be a potential time-place to 
> > > > > meet.
> > > > > Not sure in what form. If not, we can organize one in our San Ramon 
> > > > > office
> > > > > later on.
> > > > > - The effort may take a few months depending on the contributors 
> > > > > schedules.
> > > > > Would it make sense to open a branch for the ConsensusNode work?
> > > > > - APIs and the implementation of the Coordination Engine should be a 
> > > > > fairly
> > > > > independent, so it may be reasonable to add it directly to Hadoop 
> > > > > Common
> > > > > trunk.
> > > > > 
> > > > > Thanks,
> > > > > --Konstantin

[jira] [Created] (HADOOP-10822) Refactor HTTP proxyuser support out of HttpFS into common

[Alejandro Abdelnur (JIRA)]

Alejandro Abdelnur created HADOOP-10822:
---

 Summary: Refactor HTTP proxyuser support out of HttpFS into common
 Key: HADOOP-10822
 URL: https://issues.apache.org/jira/browse/HADOOP-10822
 Project: Hadoop Common
  Issue Type: Improvement
  Components: security
Affects Versions: 2.4.1
Reporter: Alejandro Abdelnur
Assignee: Alejandro Abdelnur


HttpFS implements HTTP proxyuser support inline in httpfs code.

For HADOOP-10698 we need similar functionality for KMS.

Not to duplicate code, we should refactor existing code to common.

We should also leverage HADOOP-10817.



--
This message was sent by Atlassian JIRA
(v6.2#6252)