Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

For more details, see https://builds.apache.org/job/hadoop-trunk-win/519/ [Jul 4, 2018 8:41:10 PM] (nanda) HDDS-212. Introduce NodeStateManager to manage the state of Datanodes in -1 overall The following subsystems voted -1: compile mvninstall pathlen unit The following subsystems

答复: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

+1. -邮件原件- 发件人: Subru Krishnan [mailto:su...@apache.org] 发送时间: 2018年7月6日 5:59 收件人: Wei-Chiu Chuang 抄送: Rohith Sharma K S; Subramaniam Krishnan; yarn-...@hadoop.apache.org; Hdfs-dev; Hadoop Common; mapreduce-...@hadoop.apache.org 主题: Re: [VOTE] reset/force push to clean up inadvertent

[jira] [Created] (HDFS-13723) Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh

Siyao Meng created HDFS-13723: - Summary: Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh Key: HDFS-13723 URL: https://issues.apache.org/jira/browse/HDFS-13723

[jira] [Created] (HDDS-232) Parallel unit test execution for HDDS/Ozone

Arpit Agarwal created HDDS-232: -- Summary: Parallel unit test execution for HDDS/Ozone Key: HDDS-232 URL: https://issues.apache.org/jira/browse/HDDS-232 Project: Hadoop Distributed Data Store

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

+1 on Non-routable IP idea. My preference is to start in Hadoop-common to minimize the scope and incrementally improve. However, this will be incompatible change for initial user experience on public cloud. What would be the right release vehicle for this work (3.2+ or 4.x)? Regards, Eric

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

Unfortunately since it was a merge commit, less straightforward to revert. You can find the details in the original mail thread: http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201807.mbox/%3CCAHqguubKBqwfUMwhtJuSD7X1Bgfro_P6FV%2BhhFhMMYRaxFsF9Q%40mail.gmail.com%3E On Thu, Jul 5, 2018 at

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

I'm sorry I come to this thread late. Anu commented on INFRA-16727 saying he reverted the commit. Do we still need the vote? Thanks On Thu, Jul 5, 2018 at 2:47 PM Rohith Sharma K S wrote: > +1 > > On 5 July 2018 at 14:37, Subru Krishnan wrote: > > > Folks, > > > > There was a merge commit

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

+1 On Fri, 6 Jul 2018 at 3:07 AM, Subru Krishnan wrote: > Folks, > > There was a merge commit accidentally pushed to trunk, you can find the > details in the mail thread [1]. > > I have raised an INFRA ticket [2] to reset/force push to clean up trunk. > > Can we have a quick vote for INFRA

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

+1 On 5 July 2018 at 14:37, Subru Krishnan wrote: > Folks, > > There was a merge commit accidentally pushed to trunk, you can find the > details in the mail thread [1]. > > I have raised an INFRA ticket [2] to reset/force push to clean up trunk. > > Can we have a quick vote for INFRA sign-off

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

+1 for this. - Sunil On Thu, Jul 5, 2018 at 2:37 PM Subru Krishnan wrote: > Folks, > > There was a merge commit accidentally pushed to trunk, you can find the > details in the mail thread [1]. > > I have raised an INFRA ticket [2] to reset/force push to clean up trunk. > > Can we have a quick

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

+1 On 7/5/18, 2:44 PM, "Giovanni Matteo Fumarola" wrote: +1 On Thu, Jul 5, 2018 at 2:41 PM, Wangda Tan wrote: > +1 > > On Thu, Jul 5, 2018 at 2:37 PM Subru Krishnan wrote: > > > Folks, > > > > There was a merge commit accidentally pushed to

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

+1 On Thu, Jul 5, 2018 at 2:41 PM, Wangda Tan wrote: > +1 > > On Thu, Jul 5, 2018 at 2:37 PM Subru Krishnan wrote: > > > Folks, > > > > There was a merge commit accidentally pushed to trunk, you can find the > > details in the mail thread [1]. > > > > I have raised an INFRA ticket [2] to

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

+1 On Thu, Jul 5, 2018 at 2:37 PM Subru Krishnan wrote: > Folks, > > There was a merge commit accidentally pushed to trunk, you can find the > details in the mail thread [1]. > > I have raised an INFRA ticket [2] to reset/force push to clean up trunk. > > Can we have a quick vote for INFRA

[VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

Folks, There was a merge commit accidentally pushed to trunk, you can find the details in the mail thread [1]. I have raised an INFRA ticket [2] to reset/force push to clean up trunk. Can we have a quick vote for INFRA sign-off to proceed as this is blocking all commits? Thanks, Subru [1]

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

+1 from me as well. On Thu, Jul 5, 2018 at 5:19 PM, Steve Loughran wrote: > > > > On 5 Jul 2018, at 23:15, Anu Engineer wrote: > > > > +1, on the Non-Routable Idea. We like it so much that we added it to the > Ozone roadmap. > > https://issues.apache.org/jira/browse/HDDS-231 > > > > If there

Re: Merge branch commit in trunk by mistake

I ran “git revert -c c163d1797ade0f47d35b4a44381b8ef1dfec5b60 -m 1” that will remove all changes from Giovanni’s branch (There are 3 YARN commits). I am presuming that he can recommit the dropped changes directly into trunk. I do not know off a better way than to lose changes from his branch.

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

> On 5 Jul 2018, at 23:15, Anu Engineer wrote: > > +1, on the Non-Routable Idea. We like it so much that we added it to the > Ozone roadmap. > https://issues.apache.org/jira/browse/HDDS-231 > > If there is consensus on bringing this to Hadoop in general, we can build > this feature in

Re: Merge branch commit in trunk by mistake

Adding back hdfs/common/mr-dev again to cc list. Here's the last merge revert commit: https://github.com/apache/hadoop/commit/39ad98903a5f042573b97a2e5438bc57af7cc7a1 On Thu, Jul 5, 2018 at 2:17 PM Wangda Tan wrote: > It looks like the latest revert is not correct, many of commits get >

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

+1, on the Non-Routable Idea. We like it so much that we added it to the Ozone roadmap. https://issues.apache.org/jira/browse/HDDS-231 If there is consensus on bringing this to Hadoop in general, we can build this feature in common. --Anu On 7/5/18, 1:09 PM, "Sean Busbey" wrote: I

Re: Merge branch commit in trunk by mistake

+ hdfs-dev/common-dev/mapreduce-dev On Thu, Jul 5, 2018 at 2:09 PM Sunil G wrote: > I just see that this is reverted. > > commit 39ad98903a5f042573b97a2e5438bc57af7cc7a1 (origin/trunk, origin/HEAD) > Author: Anu Engineer > Date: Thu Jul 5 12:22:18 2018 -0700 > > Revert "Merge branch

Re: Merge branch commit in trunk by mistake

Based on conversations with Giovanni and Subru, I have pushed a revert for this merge. Thanks Anu On 7/5/18, 12:55 PM, "Giovanni Matteo Fumarola" wrote: + common-dev and hdfs-dev as fyi. Thanks Subru and Sean for the answer. On Thu, Jul 5, 2018 at 12:14 PM, Subru

[jira] [Created] (HDFS-13722) HDFS Native Client Fails Compilation on Ubuntu 18.04

Jack Bearden created HDFS-13722: --- Summary: HDFS Native Client Fails Compilation on Ubuntu 18.04 Key: HDFS-13722 URL: https://issues.apache.org/jira/browse/HDFS-13722 Project: Hadoop HDFS Issue

[jira] [Created] (HDDS-231) Exclude connections from routable/non-local subnets by default

Arpit Agarwal created HDDS-231: -- Summary: Exclude connections from routable/non-local subnets by default Key: HDDS-231 URL: https://issues.apache.org/jira/browse/HDDS-231 Project: Hadoop Distributed

[jira] [Created] (HDFS-13721) NPE in DataNode due to uninitialized DiskBalancer

Xiao Chen created HDFS-13721: Summary: NPE in DataNode due to uninitialized DiskBalancer Key: HDFS-13721 URL: https://issues.apache.org/jira/browse/HDFS-13721 Project: Hadoop HDFS Issue Type:

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

I really, really like the approach of defaulting to only non-routeable IPs allowed. it seems like a good tradeoff for complexity of implementation, pain to reconfigure, and level of protection. On Thu, Jul 5, 2018 at 2:25 PM, Todd Lipcon wrote: > The approach we took in Apache Kudu is that, if

Re: Merge branch commit in trunk by mistake

+ common-dev and hdfs-dev as fyi. Thanks Subru and Sean for the answer. On Thu, Jul 5, 2018 at 12:14 PM, Subru Krishnan wrote: > Looking at the merge commit, I feel it's better to reset/force push > especially since this is still the latest commit on trunk. > > I have raised an INFRA ticket

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

The approach we took in Apache Kudu is that, if Kerberos hasn't been enabled, we default to a whitelist of subnets. The default whitelist is 127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,169.254.0.0/16 which matches the IANA "non-routeable IP" subnet list. In other words, out-of-the-box,

[jira] [Created] (HDDS-230) Ozone Datanode exits during data write through Ratis

Mukul Kumar Singh created HDDS-230: -- Summary: Ozone Datanode exits during data write through Ratis Key: HDDS-230 URL: https://issues.apache.org/jira/browse/HDDS-230 Project: Hadoop Distributed Data

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

Hadoop default configuration aimed for user friendliness to increase adoption, and security can be enabled one by one. This approach is most problematic to security because system can be compromised before all security features are turned on. Larry's proposal will add some safety to remind

[jira] [Created] (HDDS-229) Remove singleton for Handler

Bharat Viswanadham created HDDS-229: --- Summary: Remove singleton for Handler Key: HDDS-229 URL: https://issues.apache.org/jira/browse/HDDS-229 Project: Hadoop Distributed Data Store Issue

[jira] [Created] (HDDS-228) Add the ReplicaMaps to ContainerStateManager

Anu Engineer created HDDS-228: - Summary: Add the ReplicaMaps to ContainerStateManager Key: HDDS-228 URL: https://issues.apache.org/jira/browse/HDDS-228 Project: Hadoop Distributed Data Store

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

Hi Steve - This is a long overdue DISCUSS thread! Perhaps the UIs can very visibly state (in red) "WARNING: UNSECURED UI ACCESS - OPEN TO COMPROMISE" - maybe even force a click through the warning to get to the page like SSL exceptions in the browser do? Similar tactic for UI access without SSL?

[jira] [Created] (HDDS-227) Use Grpc as the default transport protocol for Standalone pipeline

Mukul Kumar Singh created HDDS-227: -- Summary: Use Grpc as the default transport protocol for Standalone pipeline Key: HDDS-227 URL: https://issues.apache.org/jira/browse/HDDS-227 Project: Hadoop

[jira] [Created] (HDDS-226) Client should update block length in OM while committing the key

Mukul Kumar Singh created HDDS-226: -- Summary: Client should update block length in OM while committing the key Key: HDDS-226 URL: https://issues.apache.org/jira/browse/HDDS-226 Project: Hadoop

[jira] [Created] (HDFS-13720) HDFS dataset Anti-Affinity Block Placement across DataNodes for data local task optimization (improve Spark executor utilization & performance)

Hari Sekhon created HDFS-13720: -- Summary: HDFS dataset Anti-Affinity Block Placement across DataNodes for data local task optimization (improve Spark executor utilization & performance) Key: HDFS-13720 URL:

Re: HADOOP-15558 review

Adding Shreya and other team mates to the thread Regards, Chaitanya On Thu, Jul 5, 2018 at 3:30 PM Chaitanya M V S wrote: > Hello, > > We are a group of interns at IISc, Bangalore and have tried to implement Clay > Codes in >

HADOOP-15558 review

Hello, We are a group of interns at IISc, Bangalore and have tried to implement Clay Codes in Hadoop by using the Erasure Codec pluggable codec API (HDFS-7337). Clay codes are erasure codes which have many must have properties and are

[jira] [Created] (HDDS-225) Provide docker-compose files to check the scalability of KSM

Elek, Marton created HDDS-225: - Summary: Provide docker-compose files to check the scalability of KSM Key: HDDS-225 URL: https://issues.apache.org/jira/browse/HDDS-225 Project: Hadoop Distributed Data

Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

For more details, see https://builds.apache.org/job/hadoop-trunk-win/518/ [Jul 3, 2018 8:50:11 PM] (fabbri) HADOOP-15215 s3guard set-capacity command to fail on read/write of 0 [Jul 3, 2018 9:11:52 PM] (aengineer) HDDS-175. Refactor ContainerInfo to remove Pipeline object from it. [Jul 4, 2018