It sounds great, could you please explain what are you going to change in the main Vectorizer/Preprocessor API to support it?
пт, 31 мая 2019 г. в 14:20, Алексей Платонов <aplaton...@gmail.com>: > Hi, Igniters! > Currently we don't have an ability to deploy automatically user-defined > preprocessors and vectorizers. Client's code should be deployed manually to > Ignite server nodes. > > I have an idea how to fix it. If we pass user's classloader and one of > user-defined classes from fit-level to > ComputeUtils.affinityCallWithRetries() then we wiil be able to use > GridPeerDeployAware interface to send informtation about this classloader > to server nodes. > > To support this ability we can define interfaces like these: > > public interface DeployableObject { > public List<Object> getDependencies(); > } > > and > > public interface DeployingContext { > public Class<?> userClass(); > public ClassLoader clientClassLoader(); > } > > DeployableObject will be mark for our ignite-ml final classes like trainers > or concrete preprocessors and it can be able to return all dependencies > that should be deployed to server nodes if it's needed. If these > dependencies are DeployableObjects too then depenndencies will be unfolded > recursively. Classes that isn't defined as DeployableObject will be > recognized as user-defined (NOTE: all leaf classes in our hierarchy will be > DeployableObject). > > This list of DeployableObjects will be user for define user class loader > and one of these objects will be used for passing to GridPeerDeployAware. > > So, this logic allows to pass user-defined Preprocessors and Vectorizers to > training algorithms and pipelines. > > What do you think? > > Sincerely > Alexey Platonov >