Build failed in Jenkins: Hadoop-Common-trunk #2416

See Changes: [vvasudev] YARN-4709. NMWebServices produces incorrect JSON for containers. -- [...truncated 5113 lines...] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.309

Re: [crypto][chimera] Next steps

I'm confused. None of the other PMC members has expressed whether he or she want's the see Chimera/crypto joining Apache Commons, yet we're already discussing how JNI bindings should be handled. I'd like to see: 1) a clear statement whether Chimera/crypto should become part of Apache Commons. Do w

Jenkins build is back to normal : Hadoop-Common-trunk #2415

See

Jenkins build is back to normal : Hadoop-common-trunk-Java8 #1116

See

RE: [crypto][chimera] Next steps

At this point, it has just Java interfaces only. -Original Message- From: Colin P. McCabe [mailto:cmcc...@apache.org] Sent: Tuesday, February 23, 2016 1:29 AM To: Hadoop Common Cc: Commons Developers List Subject: Re: [crypto][chimera] Next steps I would highly recommend shading this lib

RE: Introduce Apache Kerby to Hadoop

Thanks Larry for your thoughts and inputs. >> Replacing MiniKDC with kerby certainly makes sense. Thanks. >> Kerby-izing Hadoop 3 needs to be defined carefully. Fully agree. We're still working to make the relevant Kerberos support come to the ideal state, either in Kerby project or outside of i

RE: Introduce Apache Kerby to Hadoop

Thanks for the confirm and further inputs, Steve. >> the latter would dramatically reduce the cost of wire-encrypting IPC. Yes to optimize Hadoop IPC/RPC encryption is another opportunity Kerby can help with, it's possible because we may hook Chimera or AES-NI thing into the Kerberos layer by l

Build failed in Jenkins: Hadoop-common-trunk-Java8 #1115

See Changes: [szetszwo] HDFS-8578. On upgrade, Datanode should process all storage/data dirs in -- [...truncated 820 lines...] [INFO] [INFO] --- maven-surefire-plugin:2.17:test (defau

Re: [crypto][chimera] Next steps

Checksum via JNI should be done in the commons-codec project. Gary On Feb 22, 2016 3:14 PM, "Colin P. McCabe" wrote: > Hi Jochen, > > Many CPUs come with built-in support for certain cryptographic and/or > hash/checksum-related primitives. For example, modern x86 CPUs have > CRC32C implemented

Re: Looking to a Hadoop 3 release

I think starting a 3.0 alpha soon would be a great idea. As some other people commented, this would come with no compatibility guarantees, so that we can iron out any issues. Colin On Mon, Feb 22, 2016 at 1:26 PM, Zhe Zhang wrote: > Thanks Andrew for driving the effort! > > +1 (non-binding) on

Build failed in Jenkins: Hadoop-Common-trunk #2414

See Changes: [cnauroth] HADOOP-11613. Remove commons-httpclient dependency from hadoop-azure. [zhz] HADOOP-12668. Support excluding weak Ciphers in HttpServer2 through [szetszwo] HDFS-8578. On upgrade, Datanode should process all

Re: [crypto][chimera] Next steps

On Mon, Feb 22, 2016 at 2:20 PM, Steve Loughran wrote: > >> On 22 Feb 2016, at 17:28, Colin P. McCabe wrote: >> >> I would highly recommend shading this library when it is used in >> Hadoop and/or Spark, to prevent version skew problems between Hadoop >> and Spark like we have had in the past. >>

Re: [crypto][chimera] Next steps

Hi Jochen, Many CPUs come with built-in support for certain cryptographic and/or hash/checksum-related primitives. For example, modern x86 CPUs have CRC32C implemented in hardware. Currently, this must be accessed via inline assembly expressed in JNI. It is worth it... at least in the case of c

Build failed in Jenkins: Hadoop-common-trunk-Java8 #1114

See Changes: [cnauroth] HADOOP-11613. Remove commons-httpclient dependency from hadoop-azure. [zhz] HADOOP-12668. Support excluding weak Ciphers in HttpServer2 through -- [...truncated

Re: [crypto][chimera] Next steps

> On 22 Feb 2016, at 17:28, Colin P. McCabe wrote: > > I would highly recommend shading this library when it is used in > Hadoop and/or Spark, to prevent version skew problems between Hadoop > and Spark like we have had in the past. > > What is the strategy for handling JNI components? I think

Re: [crypto][chimera] Next steps

On Mon, Feb 22, 2016 at 1:26 PM, Gangumalla, Uma wrote: > > >All files should follow the Commons Maven naming scheme to make it easy to > >reach from Maven, Ivy and so on. > >This will be commons-crypto-1.0.jar for example. > Sure. Thanks Gary. We will follow the naming convention here from Commo

Re: Looking to a Hadoop 3 release

Thanks Andrew for driving the effort! +1 (non-binding) on starting the 3.0 release process now with 3.0 as an alpha. I wanted to echo Andrew's point that backporting EC to branch-2 is a lot of work. Considering that no concrete backporting plan has been proposed, it seems quite uncertain whether

Re: [crypto][chimera] Next steps

>All files should follow the Commons Maven naming scheme to make it easy to >reach from Maven, Ivy and so on. >This will be commons-crypto-1.0.jar for example. Sure. Thanks Gary. We will follow the naming convention here from Commons. Regards, Uma On 2/22/16, 1:20 PM, "Gary Gregory" wrote: >Al

Re: [crypto][chimera] Next steps

All files should follow the Commons Maven naming scheme to make it easy to reach from Maven, Ivy and so on. This will be commons-crypto-1.0.jar for example. Gary On Mon, Feb 22, 2016 at 1:06 PM, Gangumalla, Uma wrote: > >I would highly recommend shading this library when it is used in > Hadoop

Re: [crypto][chimera] Next steps

>I would highly recommend shading this library when it is used in Hadoop and/or Spark, to prevent version skew problems between Hadoop and Spark like we have had in the past. [uma]Ha. This avoids multiple jars versions issues. Agreed IMO. >I think at a minimum, we should include the version number

Jenkins build is back to normal : Hadoop-Common-trunk #2413

See

Build failed in Jenkins: Hadoop-common-trunk-Java8 #1113

See Changes: [arp] HDFS-9842. dfs.datanode.balance.bandwidthPerSec should accept friendly -- [...truncated 5523 lines...] Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxP

[jira] [Created] (HADOOP-12832) Implement unix-like 'FsShell -touch'

Gera Shegalov created HADOOP-12832: -- Summary: Implement unix-like 'FsShell -touch' Key: HADOOP-12832 URL: https://issues.apache.org/jira/browse/HADOOP-12832 Project: Hadoop Common Issue Typ

Re: [crypto][chimera] Next steps

On Mon, Feb 22, 2016 at 6:28 PM, Colin P. McCabe wrote: > What is the strategy for handling JNI components? Wrong question, IMO. Should better be: What are the reasons for using JNI components? Couldn't they be replaced? If so, that would very much enhance the long term prospects of crypto|chime

Re: [crypto][chimera] Next steps

I would highly recommend shading this library when it is used in Hadoop and/or Spark, to prevent version skew problems between Hadoop and Spark like we have had in the past. What is the strategy for handling JNI components? I think at a minimum, we should include the version number in the native

Re: Looking to a Hadoop 3 release

+1 for a release of 3.0. There are a lot of significant, compatibility-breaking, but necessary changes in this release... we've touched on some of them in this thread. +1 for a parallel release of 2.8 as well. I think we are pretty close to this, barring a dozen or so blockers. best, Colin On

Jenkins build is back to normal : Hadoop-common-trunk-Java8 #1112

See

[jira] [Created] (HADOOP-12831) FSOutputSummer NPEs in ctor if bytes per checksum set to 0

Steve Loughran created HADOOP-12831: --- Summary: FSOutputSummer NPEs in ctor if bytes per checksum set to 0 Key: HADOOP-12831 URL: https://issues.apache.org/jira/browse/HADOOP-12831 Project: Hadoop Co

Build failed in Jenkins: Hadoop-Common-trunk #2412

See Changes: [junping_du] YARN-4386. refreshNodesGracefully() should send recommission event to -- [...truncated 8331 lines...] [javadoc] Loading source files for package org.apache.hadoop

Re: [GitHub] hadoop pull request: for archive.org

Please unsubscribe me from this group On Sun, Feb 21, 2016 at 4:03 AM, h4ck3rm1k3 wrote: > GitHub user h4ck3rm1k3 opened a pull request: > > https://github.com/apache/hadoop/pull/79 > > for archive.org > > I now have access to archive.org working. > See my test case here : > http

[GitHub] hadoop pull request: for archive.org

Github user h4ck3rm1k3 commented on the pull request: https://github.com/apache/hadoop/pull/79#issuecomment-187167034 https://github.com/h4ck3rm1k3/hadoop-archive-org-bucket-fs/issues/1 this is the problem that I found. Needs more work. Need to create test cases inside of hadoop for t

Re: Introduce Apache Kerby to Hadoop

Replacing MiniKDC with kerby certainly makes sense. Kerby-izing Hadoop 3 needs to be defined carefully. As much as a JWT proponent that I am, I don't know that that taking up non-standard features such as the JWT token would necessarily serve us well. If we are talking about client side only uptak

[GitHub] hadoop pull request: for archive.org

Github user h4ck3rm1k3 commented on the pull request: https://github.com/apache/hadoop/pull/79#issuecomment-187164944 I am having some problems with further testing. Closing this PR for now until I fix and test more. --- If your project is set up for it, you can reply to this email

[GitHub] hadoop pull request: for archive.org

Github user h4ck3rm1k3 closed the pull request at: https://github.com/apache/hadoop/pull/79 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

Re: Looking to a Hadoop 3 release

> On 20 Feb 2016, at 15:34, Junping Du wrote: > > Shall we consolidate effort for 2.8.0 and 3.0.0? It doesn't sounds reasonable > to have two alpha releases to go in parallel. Is EC feature the main > motivation of releasing hadoop 3 here? If so, I don't understand why this > feature cannot l

Re: Introduce Apache Kerby to Hadoop

I've discussed this offline with Kai, as part of the "let's fix kerberos" project. Not only is it a better Kerberos engine, we can do more diagnostics, get better algorithms and ultimately get better APIs for doing Kerberos and SASL —the latter would dramatically reduce the cost of wire-encryp

Build failed in Jenkins: Hadoop-common-trunk-Java8 #1111

See Changes: [aajisaka] YARN-4707. Remove the extra char (>) from SecureContainer.md. -- [...truncated 5513 lines...] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.8

Re: [crypto][chimera] Next steps

On Feb 21, 2016 11:59 PM, "Benedikt Ritter" wrote: > > Hi again, > > 2016-02-20 12:15 GMT+01:00 Benedikt Ritter : > > > Hi, > > > > I'd like to discuss the next steps for moving the Chimera component to > > Apache Commons. So far, none of the other PMC members has expressed his or > > her thoughts