Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

[Stanislav Lukyanov]

Hi,

Unfortunately, it's not going to be near future.
It's definitely not 2.7 (which was frozen a long time ago), almost
definitely not 2.8 (because it would probably take to much time to do and
2.8 is supposed to figure out other stuff related to Java 9+ support).
You can always workaround this by providing your own classloading logic
right in the compute task, using a sophisticated class loader in
IgniteConfiguration.classLoader, etc. Otherwise, please stay tuned and
eventually this feature will be added to the roadmap.

Stan



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

[xero]

Hi,
We are also facing this issue while trying to retrieve domain objects inside
a compute task.
Do you have plans to add this feature in the near future? maybe 2.7 release? 

regards




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

[Vladimir Ozerov]

The ticket is on the radar, but not in immediate plans. The problem might
sounds simple at first glance, but we already spent considerable time on
implementation and review, because we heavily rely on classes caching, and
a lot of internal BinaryMarshaller infrastructure should be reworked to
allow for this change. Hopefully, we will have it in Apache Ignite 2.5.

On Thu, Jan 4, 2018 at 2:28 AM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Ticket is still open. Vladimir, looks like it's assigned to you. Do you
> have any plans to work on it?
>
> https://issues.apache.org/jira/browse/IGNITE-5038
>
> -Val
>
> On Wed, Jan 3, 2018 at 1:26 PM, Abeneazer Chafamo <
> chaf...@nodalexchange.com> wrote:
>
>> Is there any update on the suggested functionality to resolve cache entry
>> classes based on the caller's context first instead of relying on Ignite's
>> classloader?
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
>

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

[Valentin Kulichenko]

Ticket is still open. Vladimir, looks like it's assigned to you. Do you
have any plans to work on it?

https://issues.apache.org/jira/browse/IGNITE-5038

-Val

On Wed, Jan 3, 2018 at 1:26 PM, Abeneazer Chafamo  wrote:

> Is there any update on the suggested functionality to resolve cache entry
> classes based on the caller's context first instead of relying on Ignite's
> classloader?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

[Abeneazer Chafamo]

Is there any update on the suggested functionality to resolve cache entry
classes based on the caller's context first instead of relying on Ignite's
classloader?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

[Denis Magda]

Hi guys,


Cache entries don’t store an identification of the class loader a key or value 
was created with. This is why binary marshaller picks the system class loader 
at deserialization time by default and you get class not found exception.

>> In general, I think ignite should try to resolve a class based on the 
>> caller's context first instead of only relying on
>> ignite's classloader or what it was configured with.

That’s probably worth looking into but for the cache entries we tried to 
simplify the things - if you want to deserialize an entry then most likely 
you’re doing this on the app side that already has the class in the class path. 
For the server nodes, that usually doesn’t have the classes it’s suggest to use 
BinaryObject and BinaryObjectBuilder wrappers on top of serialized data:
https://apacheignite.readme.io/docs/binary-marshaller#binaryobject-cache-api

Vovan, what do you think on this?

—
Denis

> On Apr 24, 2017, at 12:55 AM, Dmitriy Karachentsev 
>  wrote:
> 
> Crosspost to dev list.
> 
> Igniters,
> 
> This proposal looks quite reasonable. Do we have any restrictions that
> could prevent implementing such feature?
> I think it's nice to have in Ignite.
> 
> Thanks!
> -Dmitry.
> 
> On Sun, Apr 23, 2017 at 1:37 AM, npordash  wrote:
> 
>> Thanks!
>> 
>> That would definitely help address the hack I've implemented where I have
>> to
>> reference classes in Ignite's internal package.
>> 
>> However, I still have to work with the caches in binary which is less than
>> ideal. It's pretty common in use-cases like this to first try to use
>> Thread.currentThread().getContextClassLoader() and if that's not set then
>> fallback to something else. In general, I think ignite should try to
>> resolve
>> a class based on the caller's context first instead of only relying on
>> ignite's classloader or what it was configured with.
>> 
>> In the spirit of the webinar that Denis just had, I think this kind of
>> behavior will become mandatory for the service grid to get to the point
>> where it can do deployments of services without requiring class files to be
>> on ignite's classpath. I've heard that is something that's still
>> tentatively
>> planned and the use-case I outlined is an attempt to get around the current
>> service grid limitations. :)
>> 
>> WDYT?
>> 
>> -Nick
>> 
>> 
>> 
>> --
>> View this message in context: http://apache-ignite-users.
>> 70518.x6.nabble.com/BinaryObjectImpl-deserializeValue-with-
>> specific-ClassLoader-tp12055p12171.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>> 

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

[Dmitriy Karachentsev]

Crosspost to dev list.

Igniters,

This proposal looks quite reasonable. Do we have any restrictions that
could prevent implementing such feature?
I think it's nice to have in Ignite.

Thanks!
-Dmitry.

On Sun, Apr 23, 2017 at 1:37 AM, npordash  wrote:

> Thanks!
>
> That would definitely help address the hack I've implemented where I have
> to
> reference classes in Ignite's internal package.
>
> However, I still have to work with the caches in binary which is less than
> ideal. It's pretty common in use-cases like this to first try to use
> Thread.currentThread().getContextClassLoader() and if that's not set then
> fallback to something else. In general, I think ignite should try to
> resolve
> a class based on the caller's context first instead of only relying on
> ignite's classloader or what it was configured with.
>
> In the spirit of the webinar that Denis just had, I think this kind of
> behavior will become mandatory for the service grid to get to the point
> where it can do deployments of services without requiring class files to be
> on ignite's classpath. I've heard that is something that's still
> tentatively
> planned and the use-case I outlined is an attempt to get around the current
> service grid limitations. :)
>
> WDYT?
>
> -Nick
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/BinaryObjectImpl-deserializeValue-with-
> specific-ClassLoader-tp12055p12171.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

[npordash]

Thanks!

That would definitely help address the hack I've implemented where I have to
reference classes in Ignite's internal package.

However, I still have to work with the caches in binary which is less than
ideal. It's pretty common in use-cases like this to first try to use
Thread.currentThread().getContextClassLoader() and if that's not set then
fallback to something else. In general, I think ignite should try to resolve
a class based on the caller's context first instead of only relying on
ignite's classloader or what it was configured with.

In the spirit of the webinar that Denis just had, I think this kind of
behavior will become mandatory for the service grid to get to the point
where it can do deployments of services without requiring class files to be
on ignite's classpath. I've heard that is something that's still tentatively
planned and the use-case I outlined is an attempt to get around the current
service grid limitations. :)

WDYT?

-Nick



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/BinaryObjectImpl-deserializeValue-with-specific-ClassLoader-tp12055p12171.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

[Dmitry Karachentsev]

Hi Nick,

Unfortunately there is no way to use custom class loader without hacks 
with internal API. I've opened a ticket for this feature [1], it could 
be useful in such cases, please check.


[1] https://issues.apache.org/jira/browse/IGNITE-5038

Thanks!
-Dmitry.

19.04.2017 20:09, npordash пишет:

Thanks, Dmitry!

This is using Ignite 1.9. The stack trace is pretty straight forward:



Putting things into caches works just fine (f.e. instances of that Namespace
class), but pulling them out does not since the cache is only taking
Ignite's classloader into account. For the time being I've had to resort to
a hack where read operations use a binary view of the cache and then I have
to cast entries to BinaryObjectImpl or BinaryObjectOffheapImpl in order to
access (or copy) the backing array and then delegate to
Marshaller.unumarshal(byte[], ClassLoader) so the class can be found.

-Nick



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/BinaryObjectImpl-deserializeValue-with-specific-ClassLoader-tp12055p12087.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

[npordash]

Thanks, Dmitry!

This is using Ignite 1.9. The stack trace is pretty straight forward:



Putting things into caches works just fine (f.e. instances of that Namespace
class), but pulling them out does not since the cache is only taking
Ignite's classloader into account. For the time being I've had to resort to
a hack where read operations use a binary view of the cache and then I have
to cast entries to BinaryObjectImpl or BinaryObjectOffheapImpl in order to
access (or copy) the backing array and then delegate to
Marshaller.unumarshal(byte[], ClassLoader) so the class can be found.

-Nick



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/BinaryObjectImpl-deserializeValue-with-specific-ClassLoader-tp12055p12087.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

BinaryObjectImpl.deserializeValue with specific ClassLoader

[npordash]

Hi,

I have a use-case where I'm deploying services to the grid where the service
implementation that's deployed to all ignite data nodes is really just a
skeleton that downloads implementation jar files from IGFS and initializes
the "real" service using a URLClassLoader (similar to what Storm, Flink, and
other systems do).

I'm able to do this by having a client node serialize the real service
implementation directly using the grid marshaller which gives me a byte
array, then in the skeleton service's init method, which gets started on a
data node, I use the marshaller again to deserialize the implementation
using the above mentioned URLClassLoader. I can do this using
Marshaller.unmarshal(byte[], ClassLoader).

This works great, but the problem I'm now running into is that the service
will create some caches which use classes that are only available in the
aforementioned URLClassLoader. This means whenever I try to read anything
from the cache I'll get a java.lang.ClassNotFoundException. It seems like
the issue is because there is no way to specify a ClassLoader to use from
the call-site (like in the Marshaller example above) and
BinaryObjectImpl.deserializeValue is not taking the thread context
classloader into account, it is only looking at the classloader specified in
IgniteConfiguration or it defaults to the system classloader.

Is there any way around this or do I have to resort to working with caches
using binary objects only?

Thanks!
-Nick



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/BinaryObjectImpl-deserializeValue-with-specific-ClassLoader-tp12055.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.