b%40163.com%22%5D>
> Replied Message
> From zhangjian<1361320...@qq.com> <mailto:1361320...@qq.com>
> Date 06/19/2024 11:03
> ToZhanghaobo <mailto:hfutzhan...@163.com>
> Subject Re: [Discuss] RBF: Aynchronous router RPC.
> Hi，Zhanghaobo, th
...@163.com
|
Replied Message
| From | zhangjian<1361320...@qq.com.INVALID> |
| Date | 06/6/2024 11:49 |
| To | Xiaoqiao He |
| Cc | Hadoop Common ,
Hdfs-dev ,
|
| Subject | Re: [Discuss] RBF: Aynchronous router RPC. |
Hi, xiaoqiao He, thank you so much!
Best Regards,
- zhangjian
Hi, xiaoqiao He, thank you so much!
Best Regards,
- zhangjian
> 2024年6月6日 11:33，Xiaoqiao He 写道：
>
> Hi @zhangjian <1361320...@qq.com> , the dev branch HDFS-17531 is ready now.
> FYI.
>
> Best Regards,
> - He Xiaoqiao
>
> On Tue, Jun 4, 2024 at 10:08 PM zhangjian <1361320...@qq.com> wrote:
>
Hi @zhangjian <1361320...@qq.com> , the dev branch HDFS-17531 is ready now.
FYI.
Best Regards,
- He Xiaoqiao
On Tue, Jun 4, 2024 at 10:08 PM zhangjian <1361320...@qq.com> wrote:
> Hi, Xiaoqiao He:
> Can you help create a dev branch? I don't have the permission to create
> it.
>
> Thank you
Hi, thank you all for supporting this proposal!
Recently, I will split this huge PR to small PRs.
Thank you again!
- zhangjian
> 2024年5月30日 11:57，Xiaoqiao He 写道：
>
> Great! It looks like there are no other nothing blockers.
>
> @zhangjian <1361320...@qq.com> If no other furthermore comments,
Great! It looks like there are no other nothing blockers.
@zhangjian <1361320...@qq.com> If no other furthermore comments, we should
go to the next step:
a. Create a dev branch for this proposal.
b. Split this huge PR to some small JIRA and PRs.
c. Involve some folks to review PR.
Please ping
Thanx for the details, sounds cool, good luck with the feature!!!
-Ayush
> On 29 May 2024, at 8:56 AM, zhangjian <1361320...@qq.com> wrote:
>
> Thank you for taking the time to review this proposal.
> Your opinion does point out the key issues to designing an asynchronous
> router, but my
Thank you for taking the time to review this proposal.
Your opinion does point out the key issues to designing an asynchronous router,
but my proposal can address these issues:
1. My design does not affect the functionality of existing synchronous routers
in throwing stanby or retry exceptions
Thanx folks, I had a very quick pass on the PDF and it looks good.
Maybe some doubts around the fact where it was mentioned that if a
Namenode returns a StandbyException or something on similar lines, the
Router will retry, I think we have some logic in RouterRpcClient
checking for such case, if
Sounds good. Thanks for sharing your findings.
On Sat, May 25, 2024 at 2:24 AM zhangjian <1361320...@qq.com> wrote:
> Hello everyone, I conducted a performance comparison test between sync and
> asynchronous router, and the test results showed that in single ns or multi
> ns scenarios,
Hi, any more feedback for this proposal?
> 下面是被转发的邮件：
>
> 发件人: zhangjian <1361320...@qq.com>
> 主题: 回复：[Discuss] RBF: Aynchronous router RPC.
> 日期: 2024年5月25日 GMT+8 17:23:31
> 收件人: Yuanbo Liu
> 抄送: Xiaoqiao He , Ayush Saxena ,
> inigo...@apache.org, Sangji
Hello everyone, I conducted a performance comparison test between sync and
asynchronous router, and the test results showed that in single ns or multi ns
scenarios, Asynchronous router in terms of throughput The utilization of CPU
and thread, as well as the average processing time of client
good job!
On Fri, May 24, 2024 at 1:57 AM zhangjian <1361320...@qq.com> wrote:
> Hello everyone, currently, I have tested the performance of async and sync
> router for a downstream ns:
> 1. The throughput, CPU, and thread performance of the async router are
> better than those of the sync
Hello everyone, currently, I have tested the performance of async and sync
router for a downstream ns:
1. The throughput, CPU, and thread performance of the async router are better
than those of the sync router, and its memory performance is within an
acceptable range compared to the
Great. Thanks for your addendum information.
cc @Ayush Saxena @inigo...@apache.org
Any more feedback for this proposal?
IMO The feature of asynchronous router RPC is a helpful improvement. For my
internal practice, it will improve the throughput of requests forward
significantly
and is very
Hi, Sangjin Lee, thank you for your attention. I will use my free time to do a
performance comparison recently.
> 2024年5月22日 03:42，Sangjin Lee 写道：
>
> Thanks for the great proposal, Zhangjian. On point #3, I suspect it should
> be fairly straightforward to create a small isolated synthetic
Thanks for the great proposal, Zhangjian. On point #3, I suspect it should
be fairly straightforward to create a small isolated synthetic test to
prove (or disprove) the benefits of this approach. By driving a controlled
amount of requests per second, you could see latency, memory, CPU, etc.
Hi, xiaoqiao he, thank you for your reply.
1.Currently, the server and client protocols within router can be implemented
by extends existing protocols and adding asynchronous functionality, so it will
not affect existing synchronization protocols.
Thanks for this great proposal!
Some questions after reviewing the design doc (sorry didn't review PR
carefully which is too large.)
1. This solution will involve RPC framework update, will it affect other
modules and how to
keep other modules off these changes.
2. Some RPC requests should be
Thank you for your positive attitude towards this feature. You can debug the
UTs provided in PR to better understand the current asynchronous calling
function.
> 2024年5月21日 02:04，Simbarashe Dzinamarira 写道：
>
> Excited to see this feature as well. I'll spend more time understanding the
>
Excited to see this feature as well. I'll spend more time understanding the
proposal and implementation.
On Mon, May 20, 2024 at 7:55 AM zhangjian <1361320...@qq.com.invalid> wrote:
> Hi, Yuanbo liu, thank you for your interest in this feature, I think the
> difficulty of an asynchronous router
Hi, Yuanbo liu, thank you for your interest in this feature, I think the
difficulty of an asynchronous router is not only to implement asynchronous
functions, but also to consider the readability and reusability of the code, so
as to facilitate the development of the community. I also planned
Nice to see this feature brought up. I tried to implement this feature in
our internal clusters, and know that it's a very complicated feature, CC
hdfs-dev to bring more discussion.
By the way, I'm not sure whether virtual thread of higher jdk will help in
this case.
On Mon, May 20, 2024 at 10:10
Hello everyone, currently there are some shortcomings in the RPC of HDFS router：
Currently the router's handler thread is synchronized, when the handler thread
adds the call to connection.calls, it needs to wait until the connection
notifies the call to complete, and then Only after the
24 matches
Mail list logo
