e and
>> timely fashion, let's try to address that...
This is interesting. Do you aware any means to do that? Thanks!
Regards,
Kai
-Original Message-
From: Andrew Wang [mailto:andrew.w...@cloudera.com]
Sent: Tuesday, November 21, 2017 2:22 PM
To: Zheng, Kai
Cc: Junping Du ;
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS connector
module? We have some Hadoop users that wish to have it by default for
convenience, though in the past they used it by back porting t
s we could see RC3 is being voted and
goes fine. My idea is to put the work in branch-2.9 first and expect some new
release after the 2.9.0 one. Sure let me raise it on the 2.9 release thread
when it's the right time.
Thanks Chris again for the education and the thoughts.
Regards,
Kai
There was some discussion about backporting OSS module to branch 2.x and per
Chris's suggestion we should do it in the dev list.
-Original Message-
From: Chris Douglas [mailto:cdoug...@apache.org]
Sent: Thursday, November 16, 2017 1:20 AM
To: Zheng, Kai mailto:kai.zh...@inte
Cool to have this feature! Thanks Jian and all.
Regards,
Kai
-Original Message-
From: Vinod Kumar Vavilapalli [mailto:vino...@apache.org]
Sent: Tuesday, November 07, 2017 7:20 AM
To: Jian He
Cc: yarn-...@hadoop.apache.org; common-dev@hadoop.apache.org; Hdfs-dev
; mapreduce-...@hadoop.a
Thanks Vinod.
>> Of the top of my head, one of the biggest areas is application
>> compatibility. When folks move from 2.x to 3.x, are their apps binary
>> compatible? Source compatible? Or need changes?
I thought these are good concerns from overall perspective. On the other hand,
I've discuss
eview comments in HADOOP-12756 and Genmao Yu for
addressing them, doing the complete "hadoop fs" tests.
Regards,
Kai
-Original Message-----
From: Zheng, Kai [mailto:kai.zh...@intel.com]
Sent: Wednesday, September 28, 2016 10:35 AM
To: common-dev@hadoop.apache.org
Subject: [VOTE] HADOOP-127
Could I extend this a bit longer considering the PRC holiday (during Oct 1 and
Oct 7)? If sounds good I'd like to have another week (next Wednesday) for this.
Please advise if you'd like to think otherwise. Thanks.
Regards,
Kai
-Original Message-----
From: Zheng, Kai [mai
+1 (binding) from myself.
The patch, the doc and design look good. It's a good beginning to start with
Aliyun OSS cloud. Thanks all.
Regards,
Kai
-Original Message-
From: Lei Xu [mailto:l...@cloudera.com]
Sent: Saturday, October 01, 2016 4:59 AM
To: common-dev@hadoop.apache.org;
Thank you for the catch. This should go to the common-dev mailing list.
Would you fire an issue to fix this?
Regards,
Kai
-Original Message-
From: 444...@qq.com [mailto:444...@qq.com]
Sent: Wednesday, September 28, 2016 9:10 AM
To: general
Subject: desc error on official site http://ha
Hi all,
I would like to propose a merge vote for HADOOP-12756 branch to trunk. This
branch develops support for Aliyun OSS (another cloud storage) in Hadoop.
The voting starts now and will run for 7 days till Oct 5, 2016 07:00 PM PDT.
Aliyun OSS is widely used among China's cloud users, and cur
Thanks Sammi.
My non-binding +1 to make the release candidate.
Regards,
Kai
-Original Message-
From: Chen, Sammi
Sent: Friday, September 02, 2016 4:59 PM
To: Zheng, Kai ; Andrew Wang ;
Arun Suresh
Cc: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org;
mapreduce
For the leveldb thing, wouldn't we have an alternative option in Java for the
platforms where leveldb isn't supported yet due to whatever reasons. IMO,
native library would be best to be used for optimization and production for
performance. For development and pure Java platform, by default pure
My humble feeling is almost the same regarding the urgent need of a 3.0 alpha
release.
Considering EC, shell-script rewriting and etc. are significant changes and
there are interested users that want to evaluate EC storage method, an alpha
3.0 release will definitely help a lot allowing users t
Thanks Andrew for the work around!! It works great ...
Regards,
Kai
-Original Message-
From: Andrew Wang [mailto:andrew.w...@cloudera.com]
Sent: Wednesday, July 20, 2016 8:10 AM
To: Chris Douglas
Cc: Zheng, Kai ; common-dev@hadoop.apache.org
Subject: Re: Not being able to add HDFS
Hi,
I tried many times in the week at different time but just found it's not
possible to add more HDFS contributors. I can add some Hadoop ones, though. It
becomes an issue because without adding someone and assigning issues to him
first, he won't be able to work on it and upload patches ...
C
Hi,
I noticed it's pretty hard to opt a version (say 3.0-alpha1) in the fix/target
version box in the JIRA system since the list is pretty long and not well
sorted. Could we clean it up or resort/re-list them in order for the most
possibly used ones to be displayed first.
Regards,
Kai
Yeah, this would be great, so some guys like me won't need to trouble you
asking the question again and again :). Thanks a lot.
Regards,
Kai
-Original Message-
From: Akira AJISAKA [mailto:ajisa...@oss.nttdata.co.jp]
Sent: Monday, June 20, 2016 3:17 PM
To: Zheng, Kai ; commo
22 PM
To: Zheng, Kai ; common-dev@hadoop.apache.org
Subject: Re: Different JIRA permissions for HADOOP and HDFS
Yes, the role allows committers to add/remove all the roles.
Now about 400 accounts have contributors roles in Hadoop common, and about 1000
contributors in history.
Regards,
Akira
On
mailto:ste...@hortonworks.com]
Sent: Wednesday, June 15, 2016 6:16 PM
To: Zheng, Kai
Cc: common-dev@hadoop.apache.org
Subject: Re: A top container module like hadoop-cloud for cloud integration
modules
> On 13 Jun 2016, at 14:02, Zheng, Kai wrote:
>
> Hi,
>
> Noticed it
, June 18, 2016 12:56 PM
To: Zheng, Kai ; common-dev@hadoop.apache.org
Subject: Re: Different JIRA permissions for HADOOP and HDFS
I'm doing the following steps to reduce the number of contributors:
1. Find committers who have only contributor role 2. Add them into committer
role 3. Remove
cept
>>>>HDFS.
>>>>> > >>
>>>>> > >> Please grant me also to the group.
>>>>> > >>
>>>>> > >> Thanks in advance,
>>>>> > >> -Vinay
>>>&
Hi,
Noticed it's an obvious trend Hadoop is supporting more and more cloud
platforms, I suggest we have a top container module to hold such integration
modules, like the ones for aws, openstack, azure and upcoming one aliyun. The
rational is simple besides the trend:
1. Existing modules
+1 on we would prefer more to using feature branches on big efforts. Thanks
Colin for the insights and details.
For better effect and to make feature branch more attractive, we might also
consider to give more love to such feature branches, maybe not so much as we do
to trunk. In this thinking,
se.
Regards,
Kai
-Original Message-
From: Allen Wittenauer [mailto:allenwittena...@yahoo.com]
Sent: Sunday, May 29, 2016 12:17 PM
To: Zheng, Kai
Cc: common-dev@hadoop.apache.org
Subject: Re: About fix versions
> On May 28, 2016, at 5:13 PM, Zheng, Kai wrote:
>
> Hi,
>
Hi,
This may be a stupid question but I want to make sure. What fix versions would
we fill with when a committer just wants to commit a patch to trunk or branch-2
branch? I remembered it's a release manager's role to decide which jira/patch
to include when working on a release. Would anyone hel
Thanks for thinking about this, Andrew and Zhe. I updated the patch today for
HADOOP-13010 (rather large) and would be great if we could get it in the week.
Regards,
Kai
-Original Message-
From: Zhe Zhang [mailto:zhe.zhang.resea...@gmail.com]
Sent: Friday, May 20, 2016 2:38 PM
To: Andr
It works for me now, thanks Andrew!
Regards,
Kai
-Original Message-
From: Andrew Wang [mailto:andrew.w...@cloudera.com]
Sent: Monday, May 16, 2016 12:14 AM
To: Zheng, Kai
Cc: common-dev@hadoop.apache.org
Subject: Re: Different JIRA permissions for HADOOP and HDFS
I just gave you
avoid bad state? Thanks!
-Original Message-----
From: Zheng, Kai [mailto:kai.zh...@intel.com]
Sent: Sunday, May 15, 2016 3:20 PM
To: Allen Wittenauer
Cc: common-dev@hadoop.apache.org
Subject: RE: Different JIRA permissions for HADOOP and HDFS
Thanks Allen for illustrating this in details. I
]
Sent: Saturday, May 14, 2016 9:38 AM
To: Zheng, Kai
Cc: common-dev@hadoop.apache.org
Subject: Re: Different JIRA permissions for HADOOP and HDFS
> On May 14, 2016, at 7:07 AM, Zheng, Kai wrote:
>
> Hi,
>
> Noticed this difference but not sure if it’s intended. YARN is simil
reason?
https://issues.apache.org/jira/browse/HADOOP-12782
https://issues.apache.org/jira/browse/HDFS-10285
Regards,
Kai
From: Ted Yu [mailto:yuzhih...@gmail.com]
Sent: Saturday, May 14, 2016 7:28 AM
To: Zheng, Kai
Cc: common-dev@hadoop.apache.org
Subject: Re: Different JIRA permissions for HADOOP
Hi,
Noticed this difference but not sure if it's intended. YARN is similar with
HDFS. It's not convenient. Any clarifying? Thanks. -kai
[cid:image001.png@01D1ADAF.16160940]
[cid:image002.png@01D1ADAF.16160940]
Ok, got it. Thanks for the explanation.
Regards,
Kai
From: Andrew Wang [mailto:andrew.w...@cloudera.com]
Sent: Tuesday, May 03, 2016 5:41 AM
To: Zheng, Kai
Cc: common-dev@hadoop.apache.org
Subject: Re: Release numbering for 3.x leading up to GA
>> but I'm going to spend time on o
Thanks for driving this, Andrew. Sounds great.
>> but I'm going to spend time on our first RC this week.
Sorry what does this mean? Did you mean the first RC version or 3.0.0-alpha1
will be cut out this week?
Anyway will try to get some tasks done sooner.
Regards,
Kai
-Original Message-
Looks like you missed some maven plugin. May be helpful to run `mvn install`
before the building.
-Original Message-
From: ? ? [mailto:yu20...@hotmail.com]
Sent: Tuesday, March 29, 2016 3:39 PM
To: common-dev@hadoop.apache.org
Subject: hadoop 2.7.2 build failure with error "plugin descri
Nice proposal. The high performance crypto offering is nice to be broadly
accessible. Thanks.
Non-binding +1.
Regards,
Kai
-Original Message-
From: Chris Nauroth [mailto:cnaur...@hortonworks.com]
Sent: Wednesday, March 23, 2016 5:47 AM
To: common-dev@hadoop.apache.org; Commons Develope
st,
>Andrew
>
>On Sun, Feb 28, 2016 at 11:28 PM, Zheng, Kai wrote:
>
>> Hi,
>>
>> I'm wondering if we could get rid of the style checking in getters
>> like the following (from HDFS-9733). It's annoying because it's a
>> common Java practi
and post patch to disable that checkstyle rule? You can
look at HADOOP-12713 for an example. Ping me and I'll review.
Best,
Andrew
On Sun, Feb 28, 2016 at 11:28 PM, Zheng, Kai wrote:
> Hi,
>
> I'm wondering if we could get rid of the style checking in getters
> like the fol
Hi,
I'm wondering if we could get rid of the style checking in getters like the
following (from HDFS-9733). It's annoying because it's a common Java practice
and widely used in the project.
void setBlockLocations(LocatedBlocks blockLocations) {:42: 'blockLocations'
hides a field.
void setTim
ameNode RPC calls or similar still. Throughput drops 3-4x, or
> worse.
>
> > On Feb 22, 2016, at 4:56 PM, Zheng, Kai wrote:
> >
> > Thanks for the confirm and further inputs, Steve.
> >
> >>> the latter would dramatically reduce the cost of wire-encrypting IPC
-4x, or worse.
> On Feb 22, 2016, at 4:56 PM, Zheng, Kai wrote:
>
> Thanks for the confirm and further inputs, Steve.
>
>>> the latter would dramatically reduce the cost of wire-encrypting IPC.
> Yes to optimize Hadoop IPC/RPC encryption is another opportunity Kerby can
&
ultimately get better APIs for
> doing Kerberos and SASL —the latter would dramatically reduce the cost
> of wire-encrypting IPC.
>
> For now, I'd like to see basic steps -upgrading minkdc to krypto, see
> how it works.
>
> Long term, I'd like Hadoop 3 to be Kerby-iz
ost of wire-encrypting IPC.
For now, I'd like to see basic steps -upgrading minkdc to krypto, see how it
works.
Long term, I'd like Hadoop 3 to be Kerby-ized
> On 22 Feb 2016, at 06:41, Zheng, Kai wrote:
>
> Hi folks,
>
> I'd like to mention Apache Kerby [1] here to
Hi folks,
I'd like to mention Apache Kerby [1] here to the community and propose to
introduce the project to Hadoop, a sub project of Apache Directory project.
Apache Kerby is a Kerberos centric project and aims to provide a first Java
Kerberos library that contains both client and server suppo
Thanks Andrew for driving this. Wonder if it's a good chance for HADOOP-12579
(Deprecate and remove WriteableRPCEngine) to be in. Note it's not an
incompatible change, but feel better to be done in the major release.
Regards,
Kai
-Original Message-
From: Andrew Wang [mailto:andrew.w...@
Hi,
I recently did some investigation about DistCp and have some questions. I
thought before diving into JIRA things it would be good to discuss them first
here.
I read the doc at the following link and regard it as the latest revision that
corresponds with the trunk codebase.
http://hadoop.ap
ption. As a result,
any exception that needs to cross a process boundary over RPC generally ends up
needing to subclass IOException. I don't think this is something that can be
changed easily.
--Chris Nauroth
On 12/28/15, 7:46 PM, "Zheng, Kai" wrote:
>
>Hi,
>
>Wou
Hi,
Would it be good to add to throw a subtype exception in addition to a more
general exception already there in the thrown list? Is it some coding style
that's required to follow or developers can do it as they like?
It's often seen that only the general exception is in the thrown list, like
Non-binding +1
According to our extensive performance tests, striping + ISA-L coder based
erasure coding not only can save storage, but also can increase the throughput
of a client or a cluster. It will be a great addition to HDFS and its users.
Based on the latest branch codes, we also observe
It's OK now.
Regards,
Kai
-Original Message-
From: Vinayakumar B [mailto:vinayakum...@apache.org]
Sent: Friday, June 05, 2015 3:12 PM
To: common-dev@hadoop.apache.org
Subject: Re: Jira down :(
Thanks Tsuyoshi.
Regards,
Vinay
On Fri, Jun 5, 2015 at 12:38 PM, Tsuyoshi Ozawa wrote:
> H
-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org;
yarn-...@hadoop.apache.org
Subject: Re: IMPORTANT: testing patches for branches
On Apr 22, 2015, at 11:34 PM, Zheng, Kai wrote:
> Hi Allen,
>
> This sounds great.
>
>>> Naming a patch foo-HDFS-7285.00.patch should get te
Hi Allen,
This sounds great.
>> Naming a patch foo-HDFS-7285.00.patch should get tested on the HDFS-7285
>> branch.
Does it happen locally in developer's machine when running test-patch.sh, or
also mean something in Hadoop Jenkins building when a JIRA becoming patch
available? Thanks.
Regard
Hello Leo/Liou
>> And the plugin interface can be as simple as this function (error handling
>> ignored here) ...
I thought it's good to have the pluggable allowing to customize the method how
to perform the mapping.
You could open a JIRA for this. If you'd like to work on it and need help,
pl
Might I have some comments for this, just providing my thought. Thanks.
>> If we start now, it might make it out by 2016. If we start now,
>> downstreamers can start aligning themselves to land versions that suit at
>> about the same time.
Not only for down streamers to align with the long term
Kumar Vavilapalli [mailto:vino...@hortonworks.com]
Sent: Tuesday, March 03, 2015 11:31 AM
To: Zheng, Kai
Cc: mapreduce-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; Hadoop Common;
yarn-...@hadoop.apache.org
Subject: Re: 2.7 status
Kai, please ping the reviewers that were already looking at
Is it interested to get the following issues in the release ? Thanks !
HADOOP-10670
HADOOP-10671
Regards,
Kai
-Original Message-
From: Yongjun Zhang [mailto:yzh...@cloudera.com]
Sent: Monday, March 02, 2015 4:46 AM
To: hdfs-...@hadoop.apache.org
Cc: Vinod Kumar Vavilapalli; Hadoop Commo
Sorry for the bad. I thought it was sending to my colleagues.
By the way, for the JDK8 support, we (Intel) would like to investigate further
and help, thanks.
Regards,
Kai
-Original Message-
From: Zheng, Kai
Sent: Tuesday, March 03, 2015 8:49 AM
To: common-dev@hadoop.apache.org
JDK8 support is in the consideration, looks like many issues were reported and
resolved already.
https://issues.apache.org/jira/browse/HADOOP-11090
-Original Message-
From: Andrew Wang [mailto:andrew.w...@cloudera.com]
Sent: Tuesday, March 03, 2015 7:20 AM
To: common-dev@hadoop.apache.
erage with something that more closely approximates a secured deployment.
Chris Nauroth
Hortonworks
http://hortonworks.com/
On Thu, Jun 26, 2014 at 7:27 AM, Zheng, Kai wrote:
> Hi David,
>
> Quite some time ago I opened HADOOP-9952 and planned to create secured
> MiniClusters by
e're no other concerns. We would provide the full list of
breakdown JIRAs and attach it in HADOOP-9392 then for further collaboration.
Regards,
Kai
From: larry mccay [mailto:larry.mc...@gmail.com]
Sent: Wednesday, September 18, 2013 6:27 AM
To: Zheng, Kai; Chen, Haifeng; common-dev@hadoop.apache.org
reakdown JIRAs and attach it in HADOOP-9392 then for further collaboration.
Regards,
Kai
From: larry mccay [mailto:larry.mc...@gmail.com]
Sent: Wednesday, September 18, 2013 6:27 AM
To: Zheng, Kai; Chen, Haifeng; common-dev@hadoop.apache.org
Subject: Re: [DISCUSS] Security Efforts and Branching
All -
Just updated by Harsh J. Thanks.
The trunk-win branch has long since been merged into active trunk, please don't
use it anymore. We have Windows instructions in trunk and branch-2 README.
Sorry for any convenience.
Regards,
Kai
-Original Message-
From: Zheng, Kai
Sent: Thu
Perhaps you could try the 'branch-trunk-win' branch, and look for some building
instructions in it. Not sure this works or not, though.
-Original Message-
From: Ranjan Dutta [mailto:rdbmsdata.ran...@gmail.com]
Sent: Thursday, September 05, 2013 3:43 PM
To: common-dev@hadoop.apache.org
Su
Got it Suresh.
So I guess HADOOP-9797 (and the family) for the UGI change would be a fit to
this rule right. The refactoring is improving and cleaning UGI, also preparing
for TokenAuth feature. According to this rule the changes would be in trunk
first. Thanks for your guidance.
Regards,
Kai
ably move forward with:
>> > >>>>>>>
>> > >>>>>>> 1. TokenAuth method - assuming this means the pluggable
>> > >>>>>>> authentication mechanisms within the RPC layer (2 votes:
>> > >>>
t; > >> before anything else can be started.
> > >> I happen to agree with Brian that #4 Hadoop SSO Tokens are
> > >> central to
> > all
> > >> the other components and should probably be defined and POC'd in
> > >> short order.
>
e our collaborative effort to contribute code to
these JIRAs.
Regards,
Kai
-Original Message-
From: Larry McCay [mailto:lmc...@hortonworks.com]
Sent: Thursday, July 04, 2013 4:10 AM
To: Zheng, Kai
Cc: common-dev@hadoop.apache.org
Subject: Re: [DISCUSS] Hadoop SSO/Token Server Components
Hi Larry,
Thanks for the update. Good to see that with this update we are now aligned on
most points.
I have also updated our TokenAuth design in HADOOP-9392. The new revision
incorporates feedback and suggestions in related discussion with the community,
particularly from Microsoft and other
In my view it should be for the whole ecosystem. One inspiration of this is to
ease the collaboration and discussion for the work on going about token based
authentication and SSO, which absolutely targets the ecosystem, although the
coming up libraries and facilities might reside in hadoop comm
-9477) posixGroups support for LDAP groups
mapping service
Hi Zheng
You can resubmit the patch again. Sometimes the Jenkins may not work well.
On Mon, 2013-05-06 at 07:36 +, Zheng, Kai wrote:
> Hi,
>
> Can anyone help take a look at this why the patch submitting won't trigger
>
Hi,
Can anyone help take a look at this why the patch submitting won't trigger the
checking of HADOOP-QA? Thanks.
Regards,
Kai
-Original Message-
From: Dapeng Sun (JIRA) [mailto:j...@apache.org]
Sent: Sunday, May 05, 2013 11:00 PM
To: Zheng, Kai
Subject: [jira] [Updated] (HADOOP
Hi Thomas,
My name is Kai from project Rhino team now working on the topic of "Token based
authentication and single sign on". I am glad to know that you are interested
in this topic and I'd love to work with you for that.
> Hello Devs,
> My name is Thomas and I'm currently doing my master re
Hi everyone,
Before I open a JIRA, I'd like to know how you like it, a file based group
mapping provider. The idea is as follows.
1. Have a new user group mapping provider such as FileBasedGroupMapping, which
consumes a mapping file like below:
$HADOOP_CONF/groupsMapping.txt:
group1:user1,user2
JIRA is opened for this:
https://issues.apache.org/jira/browse/HADOOP-8943
-Original Message-
From: Zheng, Kai [mailto:kai.zh...@intel.com]
Sent: Friday, October 19, 2012 10:17 AM
To: common-dev@hadoop.apache.org; na...@cloudera.com
Subject: RE: Questions and possible improvements for
types of LDAP
implementations. The defaults just happen to be geared towards Active Directory.
Thanks,
Natty
-Original Message-
From: Zheng, Kai [mailto:kai.zh...@intel.com]
Sent: Friday, October 19, 2012 8:32 AM
To: common-dev@hadoop.apache.org
Subject: Questions and possible impro
Hi All,
Regarding LdapGroupsMapping, I have following questions:
1. Is it possible to use ShellBasedUnixGroupsMapping for Hadoop service
principals/users, and LdapGroupsMapping for end user accounts?
In our environment, normal end users (along with their groups info) for Hadoop
cluster
76 matches
Mail list logo