[ 
https://issues.apache.org/jira/browse/HADOOP-5640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754841#action_12754841
 ] 

Todd Lipcon commented on HADOOP-5640:
-------------------------------------

bq. Since Thrift RPC has replaced client-NN rpc, can't you do the same for the 
DN to NN rpc? 

Currently we haven't replaced the Client-NN RPC with anything - for Java 
clients using DistributedFileSystem, it continues to use Hadoop RPC and the 
BlockXceiverProtocol. The Thrift stuff is a second option for clients not 
written in Java. Our particular use case for the Thrift interface is mostly 
concerned with management and metadata, and less with making a fully functional 
and performant client interface replacement.

bq. Look at the lifecycle Jira (HDFS-326) and explain if there is any overlap 
on the lifecycle events.

Not to be too pessimistic, but that JIRA has been in the works for over a year 
and recently got pushed back again so it won't be in 0.21. We'd like to have 
this in 0.21. If we go ahead and mark this API as Unstable as you mentioned, 
I'd be happy to do the work at a later date to cut it back out if HDFS-326 
covers the use case.

bq. Explain how you are planning to deliver the events. 

The current patch should illustrate this. The event dispatch uses a separate 
thread to make the calls on the plugins. Events are dispatched by submitting 
Runnables to this thread, which is not blocking. This puts the synchronization 
onus on the receiver, but earlier in the discussion on this JIRA we determined 
that was probably preferable to possibly allowing plugin code to block daemon 
code.

bq. Mark the APIs as Unstable for now.

Will do. I'll upload a rebased patch on trunk with this change later this week.

> Allow ServicePlugins to hook callbacks into key service events
> --------------------------------------------------------------
>
>                 Key: HADOOP-5640
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5640
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: util
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hadoop-5640.txt, hadoop-5640.txt, HADOOP-5640.v2.txt, 
> hadoop-5640.v3.txt
>
>
> HADOOP-5257 added the ability for NameNode and DataNode to start and stop 
> ServicePlugin implementations at NN/DN start/stop. However, this is 
> insufficient integration for some common use cases.
> We should add some functionality for Plugins to subscribe to events generated 
> by the service they're plugging into. Some potential hook points are:
> NameNode:
>   - new datanode registered
>   - datanode has died
>   - exception caught
>   - etc?
> DataNode:
>   - startup
>   - initial registration with NN complete (this is important for HADOOP-4707 
> to sync up datanode.dnRegistration.name with the NN-side registration)
>   - namenode reconnect
>   - some block transfer hooks?
>   - exception caught
> I see two potential routes for implementation:
> 1) We make an enum for the types of hookpoints and have a general function in 
> the ServicePlugin interface. Something like:
> {code:java}
> enum HookPoint {
>   DN_STARTUP,
>   DN_RECEIVED_NEW_BLOCK,
>   DN_CAUGHT_EXCEPTION,
>  ...
> }
> void runHook(HookPoint hp, Object value);
> {code}
> 2) We make classes specific to each "pluggable" as was originally suggested 
> in HADDOP-5257. Something like:
> {code:java}
> class DataNodePlugin {
>   void datanodeStarted() {}
>   void receivedNewBlock(block info, etc) {}
>   void caughtException(Exception e) {}
>   ...
> }
> {code}
> I personally prefer option (2) since we can ensure plugin API compatibility 
> at compile-time, and we avoid an ugly switch statement in a runHook() 
> function.
> Interested to hear what people's thoughts are here.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to