Hi, Igniters!
I've prepared a PR with fixes to correct peer class loading for ML-lambdas.
Here it is: https://github.com/apache/ignite/pull/6853
It will be great if someone reviews my PR. Alexey, Yuriy, could you please
to see this contribution.
Thanks!
Best regards.
Alexey Platonov
ср, 5 июн. 2019 г. в 17:20, Yuriy Babak <y.ch...@gmail.com>:
> Alexey,
>
> This is a cool change, do you create a ticket for it?
>
> If no I can create one.
>
> Best regards,
> Yuriy Babak
>
>
> пт, 31 мая 2019 г. в 14:20, Алексей Платонов <aplaton...@gmail.com>:
>
> > Hi, Igniters!
> > Currently we don't have an ability to deploy automatically user-defined
> > preprocessors and vectorizers. Client's code should be deployed manually
> to
> > Ignite server nodes.
> >
> > I have an idea how to fix it. If we pass user's classloader and one of
> > user-defined classes from fit-level to
> > ComputeUtils.affinityCallWithRetries() then we wiil be able to use
> > GridPeerDeployAware interface to send informtation about this classloader
> > to server nodes.
> >
> > To support this ability we can define interfaces like these:
> >
> > public interface DeployableObject {
> > public List<Object> getDependencies();
> > }
> >
> > and
> >
> > public interface DeployingContext {
> > public Class<?> userClass();
> > public ClassLoader clientClassLoader();
> > }
> >
> > DeployableObject will be mark for our ignite-ml final classes like
> trainers
> > or concrete preprocessors and it can be able to return all dependencies
> > that should be deployed to server nodes if it's needed. If these
> > dependencies are DeployableObjects too then depenndencies will be
> unfolded
> > recursively. Classes that isn't defined as DeployableObject will be
> > recognized as user-defined (NOTE: all leaf classes in our hierarchy will
> be
> > DeployableObject).
> >
> > This list of DeployableObjects will be user for define user class loader
> > and one of these objects will be used for passing to GridPeerDeployAware.
> >
> > So, this logic allows to pass user-defined Preprocessors and Vectorizers
> to
> > training algorithms and pipelines.
> >
> > What do you think?
> >
> > Sincerely
> > Alexey Platonov
> >
>
