[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028356#comment-16028356
 ] 

ASF GitHub Bot commented on IGNITE-2492:


Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1937


> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net, important
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028354#comment-16028354
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


Merged to master: {{69876116dd851d7bfbf0d82e6850654cc57d3d0b}}

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net, important
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028349#comment-16028349
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


1) I think {{CurrentAppDomain}} reflects the mechanics better: users will right 
away get a hint about underlying mechanics. {{Shared}} does not tell much, on 
the other hand.
2) Docs in code updated, readme.io ticket created: IGNITE-5330

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net, important
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Vladimir Ozerov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028276#comment-16028276
 ] 

Vladimir Ozerov commented on IGNITE-2492:
-

[~ptupitsyn], looks good to me, except of two things:
1) I would rather name the mode {{Shared}} instead of {{CurrentAppDomain}}. 
2) We need to have detailed description on how this mode works. I'e' user 
should clearly understand that loaded assembly will never be unloaded. Also, we 
should explain that potentially all nodes will have this assembly loaded to 
their address space, and explain users how to deal with different versions of 
the same assembly.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net, important
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16024659#comment-16024659
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


[~vozerov], done, please review:
* {{IgniteConfiguration.PeerAssemblyLoadingMode}} added instead of bool flag
* Use originating node first for assembly requests (include origin node id in 
peer loading object wrapper)

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net, important
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Vladimir Ozerov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023251#comment-16023251
 ] 

Vladimir Ozerov commented on IGNITE-2492:
-

[~ptupitsyn], sounds reasonable to me.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net, important
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021262#comment-16021262
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


[~vozerov], peer deploying assemblies to a separate AppDomain allows unloading, 
but has the following issues:
* Multiple AppDomains are not supported in .NET Core (Cross-platform .NET, 
IGNITE-2662)
* AppDomains do not share anything. Static variables have different values, you 
can't pass object references from one to another. So user code will observer 
unexpected behavior with static state
* AppDomains are expensive. We'll have to spin up all Ignite APIs in a separate 
domain again, register binary types, prepare reflective actions, and so on. 
Managing lifetime of these objects and JNI interop will be complicated.

Given all of the above, I propose the following:
* Replace {{IgniteConfiguration.PeerAssemblyLoadingEnabled}} with an enum 
{{IgniteConfiguration.PeerAssemblyLoadingMode}} with values {{None}} (default) 
and {{SameAppDomain}}.
* Release current implementation and implement isolated mode later.

This way users have clear understanding of what is going on and that other 
modes will be added in future.
Agreed?

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net, important
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Vladimir Ozerov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020046#comment-16020046
 ] 

Vladimir Ozerov commented on IGNITE-2492:
-

[~ptupitsyn],
I walked through the code. While this is definitely an interesting feature, 
without a clean and consistent lifecycle of a loaded domain I doubt we can 
include it into the product.

How difficult would it be to apply domain-based approach? It seems that with 
help of some fine-tuning we can force .NET to share Ignite classes between 
domains [1], so that there will be not so much cross-domain calls.

[1] 
https://stackoverflow.com/questions/15414791/full-asp-net-style-appdomain-assembly-isolation

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net, important
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010336#comment-16010336
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


[~vozerov] peer deployment implemented and enabled for Compute API. Please 
review.
Other APIs can be enabled later easily.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010333#comment-16010333
 ] 

ASF GitHub Bot commented on IGNITE-2492:


GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/1937

IGNITE-2492 .NET: Peer assembly loading



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ptupitsyn/ignite ignite-2492

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1937.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1937


commit 44f3d90e4e162460754ba4e7d2158ca25d0dbec1
Author: Pavel Tupitsyn 
Date:   2017-01-11T14:12:50Z

Adding tests

commit 60772b69cc059326b873aee8866dbf74aedeb1b1
Author: Pavel Tupitsyn 
Date:   2017-01-11T15:42:12Z

wip tests

commit f099a22bd2632cf3da2fb537fdffc533e46c25de
Author: Pavel Tupitsyn 
Date:   2017-01-11T16:18:21Z

wip tests

commit f4e036c9211c99c98a51ccafa6ff86eaec9391b4
Author: Pavel Tupitsyn 
Date:   2017-01-11T16:18:44Z

wip

commit 5ac39038ec816f81cdd960745d39fdc6d08c8efc
Author: Pavel Tupitsyn 
Date:   2017-01-11T16:29:52Z

wip tests

commit 3fdd04c0adfc13d3be1ab1066bf5f85efde5769d
Author: Pavel Tupitsyn 
Date:   2017-01-11T16:30:05Z

wip

commit 4792aa1bbb1b2440f340748f2a56f3954d34eb8b
Author: Pavel Tupitsyn 
Date:   2017-01-11T16:38:14Z

wip

commit fe20574763b494c82bd6672ac7538de4083b6d8d
Author: Pavel Tupitsyn 
Date:   2017-01-12T12:08:30Z

wip tests

commit 58a8ff78e8b8eb1d22c19cc5d6d6e8f55a07d325
Author: Pavel Tupitsyn 
Date:   2017-01-12T12:12:21Z

Test done

commit 0e371fbc9787625376d41921fb07e9d7aa2ddc74
Author: Pavel Tupitsyn 
Date:   2017-01-12T12:28:44Z

Remove current assembly resolver

commit 726ed99a4e00d6e5494e2f8abacf810483be2a94
Author: Pavel Tupitsyn 
Date:   2017-01-12T12:50:06Z

wip

commit 224f6465a6b1bb8c272fda56ecf1265ecd379388
Author: Pavel Tupitsyn 
Date:   2017-01-12T13:16:01Z

Refactor LoadedAssembliesResolver usage to SerializableObjectHolder

commit 884246e39b798e9137a1d5d6cf166babe9395fa9
Author: Pavel Tupitsyn 
Date:   2017-01-12T13:39:11Z

IsPeerAssemblyLoadingEnabled

commit cda5f9e01430f49e5961515e8fb0e6db51de0a92
Author: Pavel Tupitsyn 
Date:   2017-01-12T13:47:30Z

PeerAssemblyResolver added

commit 4ff92856a859ae8b32583a5861a46fdf02801991
Author: Pavel Tupitsyn 
Date:   2017-01-12T13:54:42Z

GetAssemblyFunc

commit f52526cc7daf6cb28e598c6e17facf18c611873f
Author: Pavel Tupitsyn 
Date:   2017-01-12T14:01:37Z

wip AssemblyLoader

commit a113edc6bd082b7bf2530b3e599244fb199b3b22
Author: Pavel Tupitsyn 
Date:   2017-01-12T14:08:44Z

wip

commit 65f5aedb212282d39355192d2d96829a9edee2cf
Author: Pavel Tupitsyn 
Date:   2017-01-12T15:19:14Z

wip

commit 068e64b84e1c2d39987ab4fdd1592db4cd55f58f
Author: Pavel Tupitsyn 
Date:   2017-01-12T15:24:10Z

wip

commit ffd896b19c8b3cca23b17944c3483b292bb73ce3
Author: Pavel Tupitsyn 
Date:   2017-01-12T15:43:00Z

wip tests

commit 75cbc6e12365c852e42652ec05a26b8b55a4fc50
Author: Pavel Tupitsyn 
Date:   2017-01-12T15:46:04Z

fix cluster

commit 1a004add3531f202fc5a82980dfbeaa879fbbf70
Author: Pavel Tupitsyn 
Date:   2017-01-12T16:02:14Z

Fix lazy instance

commit c66463a94f51df588addbc13069cd3f6f8473e6d
Author: Pavel Tupitsyn 
Date:   2017-01-12T16:18:33Z

Register system func

commit d103f134d9eee29a33fb23683b193ef48a775c29
Author: Pavel Tupitsyn 
Date:   2017-01-12T16:30:32Z

GetAssemblyByNameFunc

commit ea76139afe15543c07b2e1d24f41b00e72f74df9
Author: Pavel Tupitsyn 
Date:   2017-01-12T16:34:20Z

Test works for Serializable!

commit f75b2879919382d7316aa97113f7d48535b248b9
Author: Pavel Tupitsyn 
Date:   2017-01-12T16:41:17Z

binarizable test

commit 8f11f440dfa02bd3bb1dbee2892f3c79a7074216
Author: Pavel Tupitsyn 
Date:   2017-01-12T16:53:07Z

Fix binarizable test

commit fab4dca2715c2ba8d257e3729f08045dfe661f98
Author: Pavel Tupitsyn 
Date:   2017-01-13T07:48:05Z

Update config schema

commit 733aa0f5e96745286abac34305562000c73d90e2
Author: Pavel Tupitsyn 
Date:   2017-01-13T07:50:34Z

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010184#comment-16010184
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


Transient dependency loading fixed by wrapping all objects in graph and 
subscribing to {{AssemblyResolve}} while performing assembly loading and object 
deserialization.

Uncovered IGNITE-5213 during testing, will fix before continuing with this one.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16008674#comment-16008674
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


Added more tests with indirect type dependencies, where required 
types/assemblies are known only at deserialization time, not before.
Turns out {{PeerLoadingObjectHolder}} on the top level of object graph is not 
enough, we must wrap every user object down the graph with it.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998482#comment-15998482
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


Ok, makes sense.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Vladimir Ozerov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998351#comment-15998351
 ] 

Vladimir Ozerov commented on IGNITE-2492:
-

[~ptupitsyn], I would not enable it by default for two reasons. First, this is 
rather big and complex change which may have stability issues in it's first 
version. Should we enable it by default, negative impact might be very huge and 
we will be forced to do urgent patch release. Second, as we cannot unload 
assemblies, it could cause subtle memory leaks.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996988#comment-15996988
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


[~vozerov] what do you think, should we enable peer loading by default? I 
understand that this is often not recommended for production usage, and it is 
false in Java. But we can say the same about default Multicast IP finder. It is 
default because it does not require any setup from the user.

So I think having peer deployment enabled by default will simplify life for new 
users - POCs, demos, examples, and so on.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996551#comment-15996551
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


[~vozerov] I agree. Peer assembly loading should not be implemented within 
binary marshaller, but only enabled for particular cases. We can have a common 
wrapper object, {{PeerLoadingObjectHolder}}, which holds assembly information 
along with the object itself, and perform peer requests during deserialization 
if needed. This wrapper will be used only when peer loading is enabled and in 
places where it is applicable.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Vladimir Ozerov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996342#comment-15996342
 ] 

Vladimir Ozerov commented on IGNITE-2492:
-

[~ptupitsyn], I'd like to mention that data and code should not be mixed as a 
part of this feature implementation. That is, data layer is already well 
abstracted with help of binary marshalling. Peer-assembly-loading should be 
used only for code (compute, services, listeners, etc.). We can start with 
compute component first.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993539#comment-15993539
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


Uncovered IGNITE-5135 while fixing tests. Will fix it to simplify peer 
deployment issue diagnostics.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988872#comment-15988872
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


Merged from master, resolved conflicts, stabilized tests.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net
> Fix For: 2.1
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826063#comment-15826063
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


* Added synchronization to all assembly loading calls (on a common object in 
Marshaller) so we do not send the same assembly more than once.
* Added global concurrent map of assemblies so we don't load same byte array 
twice.

Need to make sure that transitive dependencies are loaded properly.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net, roadmap
> Fix For: 2.0
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15824296#comment-15824296
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


Issue uncovered: multiple calls to Assembly.Load from the same byte array 
causes multiple assemblies to be loaded. This wastes memory, and types from 
different instances of the same assembly can not be cast to each other.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net, roadmap
> Fix For: 2.0
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15823887#comment-15823887
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


Binarizable and Serializable support added. Another thing to consider is 
{{IBinaryObject.Deserialize}}.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net, roadmap
> Fix For: 2.0
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15821464#comment-15821464
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


This ticket is related (but no blocked by) IGNITE-2703. 
Here, with Binarizable types, during deserialization we just ask Marshaller for 
a descriptor as usual, and trigger peer loading if the type is not found in 
current assemblies.
After IGNITE-2703 is merged there will be no extra work: we will be able to run 
user code on remote nodes without binary configuration and assemblies.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net, roadmap
> Fix For: 2.0
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15820714#comment-15820714
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


Initial implementation proposal:
* Do not bother with unloading. Load assemblies in current AppDomain.
* Load assemblies by typeId. If we get an object with unknown typeId, send peer 
loading request with that typeId.
* Use existing .NET mechanisms to send requests to other nodes (send a closure 
to all nodes one by one)

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net, roadmap
> Fix For: 2.0
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15820647#comment-15820647
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


I think we should not bother with unloading assemblies and a separate AppDomain 
either.
User code requires {{IIgnite}} instance and all the nested things; this will 
cause a lot of overhead to spin up in a separate domain and may cause 
unexpected effects.

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net, roadmap
> Fix For: 2.0
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-2492) .NET: Peer assembly loading

[Pavel Tupitsyn (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15820642#comment-15820642
 ] 

Pavel Tupitsyn commented on IGNITE-2492:


* Peer class loading in Java is handled in {GridDeploymentClassLoader}
* Classes are sent between nodes via direct node messages ({GridIoManager.send})
* Classes are never unloaded

> .NET: Peer assembly loading
> ---
>
> Key: IGNITE-2492
> URL: https://issues.apache.org/jira/browse/IGNITE-2492
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>  Labels: .net, roadmap
> Fix For: 2.0
>
>
> Similar to peer class loading in Java, we can provide a possibility to load 
> assemblies on already started nodes, so that a node can execute jobs that are 
> not present on other nodes.
> Considerations:
> * Can we unload assemblies after use to free memory? This requires a separate 
> AppDomain, can we work with that?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)