Adding Helix Dev Hi Ajinkya
Thank you for the explanation. Let me explain my requirement. [1] is the correct task registry configuration in a working participant code. All the transition call backs are registered in this code. The problem here is, we have to bundle binaries of all the tasks into a single Participant. If we want to change one task, we need to rebuild the participant with other tasks as well. What I thought is, why can't we build Participants that only do specific set of tasks. For example, in the cluster there are 3 participants, Participant 1 [2], Participant 2 [3] and Participant 3 [4]. If we want to change the DataCollecting task, we only need to re build Participant 2. I got above error, once I run 3 participants in above configuration. I can understand this is due to the missing transition callback of CommandTask in Participant 1 as I have purposely commented out it. What I need to know is that, is this configuration allowed in Helix architecture or do we have to implement Participants that contain all the task implementations as in [1]. Workflow configuration for both scenarios is here [5]. [1] https://gist.github.com/DImuthuUpe/41c0db579e7d86d101d112f07ed6ea00 [2] https://gist.github.com/DImuthuUpe/ec72df1ec3207ce2dce88ff7f1756da4 [3] https://gist.github.com/DImuthuUpe/4d193a3dff3008315efa2e31f6209cac [4] https://gist.github.com/DImuthuUpe/872c432935b8d33944dd571b3ac4207b [5] https://gist.github.com/DImuthuUpe/f61851b68b685b8d6744689dc130babd Thanks Dimuthu On Fri, Nov 17, 2017 at 3:21 AM, Ajinkya Dhamnaskar <[email protected]> wrote: > Hey Dlmuthu, > > Not an expert in Helix, but from exceptions it seems, system is entering > in a state not expected by reflection. I feel https://github.com/apache > /helix/blob/master/helix-core/src/main/java/org/apache/ > helix/messaging/handling/HelixStateTransitionHandler.java#L295 is > triggering this exception. > As mentioned in the later part of the stack trace and from Helix Apache > Docs <http://helix.apache.org/0.7.1-docs/tutorial_agent.html> *("Helix is > built on the following assumption: if your distributed resource is modeled > by a finite state machine, then Helix can tell participants when they > should transition between states. In the Java API, this means implementing > transition callbacks. In the Helix agent API, this means providing commands > than can run for each transition"),* did you implement *transition > callback* for these tasks? > > On Thu, Nov 16, 2017 at 10:01 AM, DImuthu Upeksha < > [email protected]> wrote: > >> Hi Devs, >> >> I'm working on the technology evaluation to re architecture Apache >> Airavata task execution framework and Helix seems like a good candidate for >> that as it has an in built distributed generic workflow execution >> capability. After going through several tutorials, I tried to implement a >> simple workflow on Helix to demonstrate following transition >> >> Data Collecting Job -> Command Executing Job -> Data Uploading Job >> >> I managed to implement this using a Participant node that includes all >> the tasks required for above workflow. However my goal is to implement >> specialized Participants for each Job type. For example, Participant 1 >> knows only about the tasks to perform Data Collecting Job and Participant >> knows only about the task to perform Command Executing Job. When I tried to >> implement such Participants, I got following error from Participant 1. I >> can share the code samples that I have tried but before that I need to know >> whether my approach is compatible with Helix's design? Does Helix require >> all the Participants to be homogeneous? >> >> Executing data collecting >> 675892 [pool-2-thread-8] ERROR org.apache.helix.messaging.han >> dling.HelixStateTransitionHandler - Exception while executing a state >> transition task Airavata_Workflow_commandExecuteJob_0 >> java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce >> ssorImpl.java:62) >> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe >> thodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:498) >> at org.apache.helix.messaging.handling.HelixStateTransitionHand >> ler.invoke(HelixStateTransitionHandler.java:344) >> at org.apache.helix.messaging.handling.HelixStateTransitionHand >> ler.handleMessage(HelixStateTransitionHandler.java:290) >> at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:84) >> at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:41) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool >> Executor.java:1149) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo >> lExecutor.java:624) >> at java.lang.Thread.run(Thread.java:748) >> Caused by: java.lang.IllegalStateException: No callback implemented for >> task COMMAND >> at org.apache.helix.task.TaskStateModel.startTask(TaskStateMode >> l.java:302) >> at org.apache.helix.task.TaskStateModel.onBecomeRunningFromInit >> (TaskStateModel.java:69) >> ... 12 more >> 675900 [pool-2-thread-8] ERROR >> org.apache.helix.participant.statemachine.StateModel >> - Default rollback method invoked on error. Error Code: ERROR >> 675908 [pool-2-thread-8] ERROR org.apache.helix.messaging.handling.HelixTask >> - Message execution failed. msgId: 8d9a08c9-0ae7-419a-8940-6b8e6e7c1ecb, >> errorMsg: java.lang.reflect.InvocationTargetException >> 675922 [pool-2-thread-8] ERROR org.apache.helix.messaging.han >> dling.HelixStateTransitionHandler - Skip internal error. errCode: >> ERROR, errMsg: null >> >> Thanks >> Dimuthu >> > > > > -- > Thanks and regards, > > Ajinkya Dhamnaskar > Student ID : 0003469679 > Masters (CS) > +1 (812) 369- 5416 <(812)%20369-5416> >
