Re: [DISCUSS] Migrate hadoop from log4j1 to log4j2

2022-01-20 Thread Duo Zhang
t; wrote: > > > > Reload4J has fixed all of those CVEs without requiring an upgrade. > > > >> On Jan 20, 2022, at 5:56 AM, Duo Zhang wrote: > >> > >> There are 3 new CVEs for log4j1 reported recently[1][2][3]. So I think > it > >> is time t

Re: [DISCUSS] Migrate hadoop from log4j1 to log4j2

2022-01-20 Thread Duo Zhang
trics come to mind. There may be others. > > Thanks, > Arpit > > > > On Jan 20, 2022, at 5:55 AM, Duo Zhang wrote: > > > > There are 3 new CVEs for log4j1 reported recently[1][2][3]. So I think it > > is time to speed up the migration to log4j2 work[4] now. >

[DISCUSS] Migrate hadoop from log4j1 to log4j2

2022-01-20 Thread Duo Zhang
There are 3 new CVEs for log4j1 reported recently[1][2][3]. So I think it is time to speed up the migration to log4j2 work[4] now. You can see the discussion on the jira issue[4], our goal is to fully migrate to log4j2 and the current most blocking issue is lack of the

Re: [jira] [Created] (HADOOP-18069) CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client

2022-01-07 Thread Duo Zhang
https://square.github.io/okhttp/changelog/ The latest stable release is 4.9.1 which was published at 2021.1.30 https://github.com/square/okhttp/commits/master And there are still lots of commits recently. I'm not saying we should not remove it in hadoop, just want to point out that it is still

Re: [DISCUSS] Change project style guidelines to allow line length 100

2021-05-19 Thread Duo Zhang
+1 for 100 instead of 80. I can see that some projects have already changed to 120. Akira Ajisaka 于2021年5月20日周四 上午10:01写道: > I'm +1 to allow <= 100 chars. > > FYI: There were some discussions long before: > - >

Re: Status of Hadoop 2.9.x

2019-12-26 Thread Duo Zhang
Yes, HBase is proposing to bump hadoop versions for new release lines, so we want to know which release lines are still alive for hadoop. Thanks. Wei-Chiu Chuang 于2019年12月26日周四 下午6:59写道: > Hi! > > Duo found out that the last 2.9.x release, 2.9.2 was GA in Nov 2018, which > is a little more

Re: [DISCUSS] Remove Ozone and Submarine from Hadoop repo

2019-10-24 Thread Duo Zhang
+1 Akira Ajisaka 于2019年10月24日周四 下午3:21写道: > Hi folks, > > Both Ozone and Apache Submarine have separate repositories. > Can we remove these modules from hadoop-trunk? > > Regards, > Akira >

Re: [DISCUSS] About creation of Hadoop Thirdparty repository for shaded artifacts

2019-09-27 Thread Duo Zhang
For HBase we have a separated repo for hbase-thirdparty https://github.com/apache/hbase-thirdparty We will publish the artifacts to nexus so we do not need to include binaries in our git repo, just add a dependency in the pom.

Re: [NOTICE] Building trunk needs protoc 3.7.1

2019-09-23 Thread Duo Zhang
The new protobuf plugin related issues have all been pushed to trunk(though I think we'd better port them to all active branches). So what's the next step? Shade and relocated protobuf? HBase has already done this before so I do not think it will take too much time. If we all agree on the

Re: [NOTICE] Building trunk needs protoc 3.7.1

2019-09-20 Thread Duo Zhang
I think this one is alread in place so we have to upgrade... https://issues.apache.org/jira/browse/HADOOP-16557 Wangda Tan 于2019年9月21日周六 上午7:19写道: > Hi Vinay, > > A bit confused, I saw the HADOOP-13363 is still pending. Do we need to > upgrade protobuf version to 3.7.1 NOW or once HADOOP-13363

Re: [DISCUSS] GitHub PRs without JIRA number

2019-09-10 Thread Duo Zhang
OK, the indexing is finished [Tue Sep 10 14:25:07 UTC 2019] Finished branch indexing. Indexing took 15 min So maybe 30 minutes is better... And we really should cleanup the stale PRs and branches... 张铎(Duo Zhang) 于2019年9月10日周二 下午10:20写道: > Actually the job for testing PR is here... > &

Re: [DISCUSS] GitHub PRs without JIRA number

2019-09-10 Thread Duo Zhang
. Steve Loughran 于2019年9月10日周二 下午7:36写道: > > > On Tue, Sep 10, 2019 at 9:07 AM 张铎(Duo Zhang) > wrote: > >> Nits: You can change the jenkins job config to not trigger pre commit >> build >> for stale PRs if only the base branch is changed. By default, either the >

Re: [DISCUSS] GitHub PRs without JIRA number

2019-09-10 Thread Duo Zhang
Nits: You can change the jenkins job config to not trigger pre commit build for stale PRs if only the base branch is changed. By default, either the PR itself changed, or the base branch is changed, the branch sources plugin will both trigger a build. You can add a Change requests option, and

Re: [DISCUSS] ARM/aarch64 support for Hadoop

2019-09-04 Thread Duo Zhang
t; >> Thanx Vinay for the initiative, Makes sense to add support for different >> architectures. >> >> +1, for the branch idea. >> Good Luck!!! >> >> -Ayush >> >> > On 03-Sep-2019, at 6:19 AM, 张铎(Duo Zhang) >> wrote: >> > >> > For

Re: [DISCUSS] ARM/aarch64 support for Hadoop

2019-09-02 Thread Duo Zhang
For HBase, we purged all the protobuf related things from the public API, and then upgraded to a shaded and relocated version of protobuf. We have created a repo for this: https://github.com/apache/hbase-thirdparty But since the hadoop dependencies still pull in the protobuf 2.5 jars, our

Re: [DISCUSS] EOL 2.8 or another 2.8.x release?

2019-07-25 Thread Duo Zhang
IIRC we have a 2.8.5 release? On the download page: 2.8.5 2018 Sep 15 Wei-Chiu Chuang 于2019年7月25日周四 上午9:39写道: > The last 2.8 release (2.8.4) was made in the last May, more than a year > ago. https://hadoop.apache.org/old/releases.html > > How do folks feel about the fate of branch-2.8? During

Re: Please help to review patch of the HDFS-14585 (backport Byte

2019-06-30 Thread Duo Zhang
Thanks for your reply Sree Vaddi. As you said, in our internal hadoop branch we have already backported these changes, and also it is fine for us to upgrade to hadoop 3, in fact we have already started to test hadoop 3.1.x in our company. But here we are talking about the open source version of

Re: Jenkins precommit build for HDFS failing

2018-09-27 Thread Duo Zhang
Solved by https://issues.apache.org/jira/browse/INFRA-17068. 张铎(Duo Zhang) 于2018年9月28日周五 上午9:37写道: > I tried to upgrade the yetus version for hbase pre commit job and the > error message is the same with hadoop then. And the problem is that when > fetching the patch we get a 4xx, s

Re: Jenkins precommit build for HDFS failing

2018-09-27 Thread Duo Zhang
I tried to upgrade the yetus version for hbase pre commit job and the error message is the same with hadoop then. And the problem is that when fetching the patch we get a 4xx, so we just apply a nonexistent file... jira_http_fetch: https://issues.apache.org/jira/browse/HBASE-21233 returned 4xx

[jira] [Created] (HDFS-13756) Implement the FileSystemLinkResolver in an async way

2018-07-20 Thread Duo Zhang (JIRA)
Duo Zhang created HDFS-13756: Summary: Implement the FileSystemLinkResolver in an async way Key: HDFS-13756 URL: https://issues.apache.org/jira/browse/HDFS-13756 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-13755) Add retry support for async rpc

2018-07-20 Thread Duo Zhang (JIRA)
Duo Zhang created HDFS-13755: Summary: Add retry support for async rpc Key: HDFS-13755 URL: https://issues.apache.org/jira/browse/HDFS-13755 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-13754) Add authentication support for async rpc

2018-07-20 Thread Duo Zhang (JIRA)
Duo Zhang created HDFS-13754: Summary: Add authentication support for async rpc Key: HDFS-13754 URL: https://issues.apache.org/jira/browse/HDFS-13754 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-13753) Implement the mkdir API for AsyncDistributedFileSystem

2018-07-20 Thread Duo Zhang (JIRA)
Duo Zhang created HDFS-13753: Summary: Implement the mkdir API for AsyncDistributedFileSystem Key: HDFS-13753 URL: https://issues.apache.org/jira/browse/HDFS-13753 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-13643) Implement basic async rpc client

2018-05-30 Thread Duo Zhang (JIRA)
Duo Zhang created HDFS-13643: Summary: Implement basic async rpc client Key: HDFS-13643 URL: https://issues.apache.org/jira/browse/HDFS-13643 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Reopened] (HDFS-13640) enable ShortCircuit Read on UC block

2018-05-30 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HDFS-13640: -- > enable ShortCircuit Read on UC block > > >

[jira] [Resolved] (HDFS-13640) s

2018-05-30 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HDFS-13640. -- Resolution: Invalid Accident? > s > - > > Key: HDFS-13640 >

Re: [DISCUSSION] Create a branch to work on non-blocking access to HDFS

2018-05-03 Thread Duo Zhang
ce.net> wrote: > > > Ok with you lot if a few of us open a branch to work on a > non-blocking HDFS > > client? > > > > Intent is to finish up the old issue "HDFS-9924 [umbrella] > Nonblocking HDFS > > Access". On the foot of

[jira] [Created] (HDFS-9372) Typo in DataStorage.recoverTransitionRead

2015-11-03 Thread Duo Zhang (JIRA)
Duo Zhang created HDFS-9372: --- Summary: Typo in DataStorage.recoverTransitionRead Key: HDFS-9372 URL: https://issues.apache.org/jira/browse/HDFS-9372 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-8683) Implement flow control

2015-06-28 Thread Duo Zhang (JIRA)
Duo Zhang created HDFS-8683: --- Summary: Implement flow control Key: HDFS-8683 URL: https://issues.apache.org/jira/browse/HDFS-8683 Project: Hadoop HDFS Issue Type: Sub-task Reporter

[jira] [Created] (HDFS-8671) Add client support for http2 stream channels

2015-06-25 Thread Duo Zhang (JIRA)
Duo Zhang created HDFS-8671: --- Summary: Add client support for http2 stream channels Key: HDFS-8671 URL: https://issues.apache.org/jira/browse/HDFS-8671 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-8515) Abstract a DTP/2 HTTP/2 server

2015-06-01 Thread Duo Zhang (JIRA)
Duo Zhang created HDFS-8515: --- Summary: Abstract a DTP/2 HTTP/2 server Key: HDFS-8515 URL: https://issues.apache.org/jira/browse/HDFS-8515 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-8472) Implement a new block reader that reads data over HTTP/2

2015-05-24 Thread Duo Zhang (JIRA)
Duo Zhang created HDFS-8472: --- Summary: Implement a new block reader that reads data over HTTP/2 Key: HDFS-8472 URL: https://issues.apache.org/jira/browse/HDFS-8472 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-8471) Implement read block over HTTP/2

2015-05-24 Thread Duo Zhang (JIRA)
Duo Zhang created HDFS-8471: --- Summary: Implement read block over HTTP/2 Key: HDFS-8471 URL: https://issues.apache.org/jira/browse/HDFS-8471 Project: Hadoop HDFS Issue Type: Sub-task