It sounds great, could you please explain what are you going to change in
the main Vectorizer/Preprocessor API to support it?

пт, 31 мая 2019 г. в 14:20, Алексей Платонов <aplaton...@gmail.com>:

> Hi, Igniters!
> Currently we don't have an ability to deploy automatically user-defined
> preprocessors and vectorizers. Client's code should be deployed manually to
> Ignite server nodes.
>
> I have an idea how to fix it. If we pass user's classloader and one of
> user-defined classes from fit-level to
> ComputeUtils.affinityCallWithRetries() then we wiil be able to use
> GridPeerDeployAware interface to send informtation about this classloader
> to server nodes.
>
> To support this ability we can define interfaces like these:
>
> public interface DeployableObject {
>     public List<Object> getDependencies();
> }
>
> and
>
> public interface DeployingContext {
>     public Class<?> userClass();
>     public ClassLoader clientClassLoader();
> }
>
> DeployableObject will be mark for our ignite-ml final classes like trainers
> or concrete preprocessors and it can be able to return all dependencies
> that should be deployed to server nodes if it's needed. If these
> dependencies are DeployableObjects too then depenndencies will be unfolded
> recursively. Classes that isn't defined as DeployableObject will be
> recognized as user-defined (NOTE: all leaf classes in our hierarchy will be
> DeployableObject).
>
> This list of DeployableObjects will be user for define user class loader
> and one of these objects will be used for passing to GridPeerDeployAware.
>
> So, this logic allows to pass user-defined Preprocessors and Vectorizers to
> training algorithms and pipelines.
>
> What do you think?
>
> Sincerely
> Alexey Platonov
>

Reply via email to