Job load without cluster restart is advanced feature. I think we should make basic API work first.
On Mon, Oct 26, 2015 at 12:16 PM, Yakov Zhdanov <[email protected]> wrote: > I think suggested approach should work. > > Any idea on how to add dynamic jobs loading (I would like it is added in > one of nearest releases)? I.e. I copy new library to each server then load > it with Ignite API and have new jobs available without restarting cluster. > > --Yakov > > 2015-10-20 18:46 GMT+03:00 Igor Sapego <[email protected]>: > > > Hello Guys, > > > > Recently I've been working on the IGNITE-1680: CPP: Implement basic API > for > > remote job execution https://issues.apache.org/jira/browse/IGNITE-1680. > > > > I would like to discuss the approach I came up with. > > > > In my current approach I use existing portable mechanism to serialize the > > job object so we can send it to the remote node. However in C++ unlike in > > Java or C# we do not have any out-of-box way to instantiate objects of > type > > which is unknown on the compile time. Still we have some runtime type > > information provided by the portable layer (e.g. we have type id). So we > > need to implement solution to deserialize and invoke a job object and > > serialize result of this invocation. To put it simple we are going to > need > > something like this: > > > > bool InvokeRemoteJob(int32_t jobTypeId, > > PortableReaderImpl& jobReader, > > PortableWriterImpl& resultWriter); > > > > As we can not operate with type during runtime in C++ it is proposed to > > make user to provide us with such kind of function. We can define three > > simple macros so user can to provide us with the list of the jobs like > > this: > > > > IGNITE_REMOTE_JOB_LIST_BEGIN > > IGNITE_REMOTE_JOB_DECLARE(JobType1, ReturnType1) > > IGNITE_REMOTE_JOB_DECLARE(JobType2, ReturnType2) > > ... > > IGNITE_REMOTE_JOB_DECLARE(JobTypeN, ReturnTypeN) > > IGNITE_REMOTE_JOB_LIST_END > > > > This will expand to something like that: > > > > extern "C" IGNITE_IMPORT_EXPORT > > bool InvokeRemoteJob(int32_t jobTypeId, > > PortableReaderImpl& jobReader, > > PortableWriterImpl& resultWriter) > > { > > if (jobTypeId == PortableType<JobType1>::GetTypeId()) > > { > > JobType1 job(jobReader.ReadTopObject<JobType1>()); > > ReturnType1 call_res = job.call(); > > resultWriter.WriteTopObject(call_res); > > return true; > > } > > ... > > return false; > > } > > > > Every module (dynamic library or executable) that has jobs that could be > > invoked remotely is going to need to have list of the remote jobs > defined. > > Then while we are loading modules we can lookup InvokeRemoteJob function > > for every module and store them so we can invoke jobs later. > > > > I have implemented proof of concept for this approach and proved it > > working. Now I would like to hear what do you people think about it so > > please provide your comments if any. > > > > Thanks > > > > Best Regards, > > Igor > > >
