Denis, I've just prepared PR [1] with a possible implementation that covers p.1 from your list.
Please, have a look at it. Pay attention to the test, it demonstrates manual services hot-redeployment via DeploymentSpi. Is that what you mean? [1] https://github.com/apache/ignite/pull/6060 On Tue, Feb 5, 2019 at 2:14 PM Denis Mekhanikov <dmekhani...@gmail.com> wrote: > > In general, I think, we should work on the improvements in the following > order: > > 1. Cluster availability, when services are being updated. Should be solved > by usage of the DeploymentSpi > 2. Service availability, when they are being updated. This one will > probably require introduction of new API methods like > redeploy(ServiceConfiguration). > 3. Service versioning and packaging. > > I'd like to focus on the first point. Service grid will become much more > usable and mature once we implement it. > > Denis > > вт, 5 февр. 2019 г. в 14:06, Denis Mekhanikov <dmekhani...@gmail.com>: > > > Vyacheslav, > > > > I think, we can use IgniteConfiguration#deploymentSpi for tasks and > > services. > > Or we can add an analogous property. > > > > Nik, > > > > > 1. Is it possible to change the list of deployed resources in runtime > > via built-in DeploymentSPI implementations? > > > Can I add or remove jar to(from) class-path without node(cluster) > > restart? > > Yes, this is the reason why the DeploymentSpi exists. But currently only > > compute grid can use it. > > > > > 2. Can we update service dependencies via DeploymentSPI? commons-lang, > > log4j or any other common library? > > Ideally such libraries should be loaded via app class loader at node > > startup. Otherwise the same libraries will be loaded multiple times. It > > will lead to a lot of memory leaks. > > I think, we can support loading of dependencies, but discourage users from > > doing it. The proper way should be described in the documentation, and > > warnings could be generated, if too many classes get loaded via > > DeploymentSpi. > > > > > 3. User has to execute explicit Ignite API calls(undeploy(), deploy()) > > to renew service implementation. Is it correct? > > > I think we should develop some watcher, that will be watch for a > > resource change and redeploy services automatically. > > Correct. I don't like the idea to redeploy services automatically. I > > think, redeployment should be triggered explicitly. Non-obvious actions > > should be avoided. > > > > 4. Such feature would for sure improve usability of the service grid. But > > it requires much more time and work to implement. > > I think, it's better not to expand the scope too much. Otherwise > > development will take another 6 moths. > > This is a great idea, and we will keep it in mind though. > > > > 5. Yep, we need an extensive documentation on the service deployment > > procedure. > > This feature may not be perfectly clear to users, so we need some how-tos. > > > > Denis > > > > вт, 5 февр. 2019 г. в 08:19, Nikolay Izhikov <nizhi...@apache.org>: > > > >> Hello, Denis. > >> > >> Thank you for this discussion. > >> I have a few notes: > >> > >> 1. Is it possible to change the list of deployed resources in runtime via > >> built-in DeploymentSPI implementations? > >> Can I add or remove jar to(from) class-path without node(cluster) restart? > >> > >> 2. Can we update service dependencies via DeploymentSPI? commons-lang, > >> log4j or any other common library? > >> > >> 3. User has to execute explicit Ignite API calls(undeploy(), deploy()) to > >> renew service implementation. Is it correct? > >> I think we should develop some watcher, that will be watch for a resource > >> change and redeploy services automatically. > >> > >> 4. DeploymentSPI is *node-wide* configuration. This means we change > >> classpath for all services with this SPI. > >> I think this is a huge limitation of the SPI. > >> We should provide an ability to configure service-wide classpath to our > >> users as quickly as possible. > >> It is a common feature in modern service, task executor engines. > >> > >> I think the perfect world scenario would be following: > >> > >> 1. Start a client node or connect to a cluster with thin client. > >> > >> 2. Configure service classpath with some new Ignite API. > >> The only requirement for classes - they should be available > >> locally(on client node or thin client host). > >> > >> 3. User deploy the service with some Ignite API. > >> > >> 4. After depoyment completes successfully client node can be > >> stopped. > >> All required resource to run a service should be safely stored in > >> cluster and deployed to any new node. > >> > >> 5. I think we should develop examples for a DeploymentSPI usage. > >> As far as I can see, there is no such examples in our codebase for now. > >> Is it correct? If so, I will create a ticket to create such examples. > >> > >> В Вт, 05/02/2019 в 01:08 +0300, Vyacheslav Daradur пишет: > >> > Denis, thank you for driving of Service Grid's development! > >> > > >> > Sounds like a good plan. Does it mean that a user will have to > >> > register a classloader for service's class explicitly in case of using > >> > the feature? > >> > > >> > On Mon, Feb 4, 2019 at 4:38 PM Denis Mekhanikov <dmekhani...@gmail.com> > >> wrote: > >> > > > >> > > Igniters, > >> > > > >> > > I'd like to start a dedicated thread for discussion of the design of > >> > > services hot redeployment. The previous service design discussion can > >> be > >> > > found in the following thread: [1] > >> > > > >> > > Currently adding a new service or implementation change of an > >> existing one > >> > > requires restarting the hosting nodes. Service instances are > >> deserialized > >> > > using an application class loader, so the service class should be > >> present > >> > > on the classpath of the node. The only way to change the set of > >> available > >> > > classes is to restart the node. Potentially the whole cluster restart > >> can > >> > > be required. This is a major drawback in the current design. This > >> problem > >> > > should be addressed first. > >> > > > >> > > At the same time this problem can be resolved by relatively simple > >> code > >> > > changes. We need to change the way services are deserialized, and use > >> a > >> > > mechanism, that allows dynamic class changes. Deployment SPI [2] > >> seems to > >> > > be suitable for this. We can apply the same approach, which is used > >> for > >> > > tasks, so services will become dynamically modifiable. > >> > > > >> > > With this approach user will still need to perform a cancel-deploy > >> routine > >> > > for the changed service. But even with that the usability improvement > >> will > >> > > be huge. We'll think about service availability improvement after the > >> first > >> > > part is finished. > >> > > > >> > > Thoughts? > >> > > > >> > > [1] > >> > > > >> http://apache-ignite-developers.2346864.n4.nabble.com/Service-versioning-td20858.html > >> > > [2] https://apacheignite.readme.io/docs/deployment-spi#deploymentspi > >> > > > >> > > Denis > >> > > >> > > >> > > >> > -- > >> > Best Regards, Vyacheslav D. > >> > > -- Best Regards, Vyacheslav D.
