Hello!
I would like to start with a reproducer. Rafael, can you please throw
together a runnable stand-alone reproducer of this issue?
Thanks,
--
Ilya Kasnacheev
чт, 2 июл. 2020 г. в 17:28, Denis Magda <dma...@apache.org>:
> Ilya,
>
> Yes, Rafael confirmed that the peer-class-loading is set.
>
> Still does not work, this time complaining that the MyFilter class is not
> found.
> PeerClassLoading is active as far as I can tell. At least in both
> server/client ignite config
>
> Is this a known limitation or regression? Should we file an feature
> request or bug?
>
> Denis
>
> On Thursday, July 2, 2020, Ilya Kasnacheev <ilya.kasnach...@gmail.com>
> wrote:
>
>> Hello!
>>
>> Did you enable peer class loading? You need to either enable peer class
>> loading, or put your filter class(es) to all server nodes.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> чт, 2 июл. 2020 г. в 15:09, Rafael Troilo <rafael.tro...@heigit.org>:
>>
>>> Hi Denis,
>>>
>>> thank you for your quick response. Unfortunately even avoiding lambdas
>>> or anonymous classes didn't solve this problem.
>>> And was my first thought for this problem too.
>>>
>>> With:
>>>
>>> MyFilter.java
>>>
>>> import org.apache.ignite.lang.IgniteClosure;
>>>
>>> public class MyFilter<T> implements IgniteClosure<T, Long> {
>>>
>>> private static final long serialVersionUID = 1L;
>>> private final long l;
>>>
>>> public MyFilter(long l) {
>>> this.l = l;
>>> }
>>>
>>> @Override
>>> public Long apply(T e) {
>>> return l;
>>> }
>>>
>>> }
>>>
>>>
>>> and
>>>
>>> try (Ignite ignite = Ignition.start("ignite.xml")) {
>>> IgniteCache<Integer, String> cache =
>>> ignite.getOrCreateCache("ScanTransformTest");
>>> ...
>>> // 2 Running the ScanQuery on the remote server. Works!
>>> compute.broadcast(() -> {
>>> try (QueryCursor<Long> cursor = cache.query(new
>>> ScanQuery<>().setLocal(true), new MyFilter<>(234))) {
>>> for (Long row : cursor) {
>>> return row;
>>> }
>>> }
>>> return 0;
>>> })
>>>
>>> // 3 Still does not work.
>>> try (QueryCursor<Long> cursor = cache.query(new ScanQuery<>(), new
>>> MyFilter<>(345L))) {
>>> for (Long row : cursor) {
>>> System.out.println(row);
>>> break;
>>> }
>>> }
>>> }
>>>
>>> Still does not work, this time complaining that the MyFilter class is
>>> not found.
>>> PeerClassLoading is active as far as I can tell. At least in both
>>> server/client ignite config
>>> ...
>>> <property name="peerClassLoadingEnabled" value="true"/>
>>> ...
>>> it is enabled.
>>>
>>> I'm also not sure why the same class (IgniteClosure) works in the other
>>> case but not in the Transformer or whats the differnt in case 2 to 3.
>>> I guess making the MyFilter class available on the server will solve
>>> this problem, but should not peerClassLoading work here?
>>>
>>> Is there any information I can provide or test I should do to help to
>>> solve this problem.
>>>
>>> Thank you very much.
>>>
>>>
>>> Stack Trace:
>>> Exception in thread "main" javax.cache.CacheException: class
>>> org.apache.ignite.IgniteCheckedException: MyFilter
>>> at
>>> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
>>> at
>>> org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
>>> at
>>> org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$5.onHasNext(GridCacheDistributedQueryManager.java:645)
>>> at
>>> org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
>>> at
>>> org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
>>> at
>>> org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
>>> at ScanQueryTest.main(ScanQueryTest.java:57)
>>> Caused by: class org.apache.ignite.IgniteCheckedException: MyFilter
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10310)
>>> at
>>> org.apache.ignite.internal.processors.cache.query.GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
>>> at
>>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
>>> at
>>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
>>> at
>>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
>>> at
>>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
>>> at
>>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
>>> at
>>> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1843)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1468)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1365)
>>> at
>>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>>> at
>>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>>> at java.base/java.lang.Thread.run(Thread.java:834)
>>> Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
>>> MyFilter
>>> at
>>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
>>> at
>>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
>>> at
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1757)
>>> at
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> at
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:307)
>>> at
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:101)
>>> at
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:81)
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10304)
>>> ... 14 more
>>> Caused by: java.lang.ClassNotFoundException: MyFilter
>>> at
>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>>> at
>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>>> at
>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>>> at java.base/java.lang.Class.forName0(Native Method)
>>> at java.base/java.lang.Class.forName(Class.java:398)
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
>>> at
>>> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
>>> at
>>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
>>> ... 21 more
>>>
>>>
>>>
>>> On 7/1/20 7:09 PM, Denis Magda wrote:
>>> > Hi Rafael,
>>> >
>>> > Lamdas and anonymous classes capture and serialize their
>>> Inner/OuterClass
>>> > with all the fields, and classes of the fields of those fields, etc. In
>>> > your case, ScanQueryClass was captured during the lamda serialization
>>> and
>>> > that class was not found during deserialization on the other side:
>>> >
>>> > Caused by: class org.apache.ignite.binary.BinaryObjectException:
>>> Failed to
>>> > read field [name=capturingClass]
>>> > at
>>> >
>>> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
>>> > at
>>> >
>>> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
>>> > ... 20 more
>>> > Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
>>> > ScanQueryTest
>>> > at
>>> >
>>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
>>> > at
>>> >
>>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
>>> > at
>>> >
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1638)
>>> > at
>>> >
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1575)
>>> > at
>>> >
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1552)
>>> > at
>>> >
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.readClass(BinaryReaderExImpl.java:383)
>>> > at
>>> >
>>> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
>>> > at
>>> >
>>> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
>>> > at
>>> >
>>> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
>>> > ... 21 more
>>> > *Caused by: java.lang.ClassNotFoundException: ScanQueryTest*
>>> > at
>>> >
>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>>> > at
>>> >
>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>>> > at
>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>>> > at java.base/java.lang.Class.forName0(Native Method)
>>> > at java.base/java.lang.Class.forName(Class.java:398)
>>> > at
>>> >
>>> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
>>> > at
>>> >
>>> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
>>> > at
>>> >
>>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
>>> > ... 29 more
>>> >
>>> > My advice is to avoid using lambdas and anonymous classes due to this
>>> > specificity. Instead, you can create "private static classes" and
>>> Ignite
>>> > will be serializing an instance of that class only:
>>> >
>>> > private static class MyFilter implements IgniteClosure {
>>> > ....
>>> > }
>>> >
>>> > Lamdas are nice for demos and simple code samples where
>>> Inner/OuterClasses
>>> > are simple Java classes. While in real production environments, things
>>> can
>>> > get messy.
>>> >
>>> > -
>>> > Denis
>>> >
>>> >
>>> > On Wed, Jul 1, 2020 at 9:36 AM Rafael Troilo <rafael.tro...@heigit.org
>>> >
>>> > wrote:
>>> >
>>> >> Hey Guys,
>>> >>
>>> >> I have a strange problem with the Transform part of a ScanQuery that
>>> I get
>>> >> a deserialize object exception for the SerializedLambda.
>>> >>> javax.cache.CacheException: class
>>> >> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
>>> >> [typeName=java.lang.invoke.SerializedLambda]
>>> >>
>>> >> Strange is that it works very well in other cases like:
>>> >>
>>> >> try (Ignite ignite = Ignition.start("ignite.xml")) {
>>> >> IgniteCache<Integer,String> cache =
>>> >> ignite.createCache("ScanTransformTest");
>>> >> cache.put(1, "Hello");
>>> >>
>>> >> // 1 Just a broadcast with a IgniteClosure works!
>>> >> compute.broadcast((a) -> 123L,
>>> "Hello").forEach(System.out::println);
>>> >>
>>> >> // 2 Running the ScanQuery on the remote server. Works too!
>>> >> compute.broadcast(() -> {
>>> >> try(QueryCursor<Long> cursor = cache.query(new
>>> >> ScanQuery<>().setLocal(true), (a) -> 234L)){
>>> >> for( Long row : cursor) {
>>> >> return row;
>>> >> }
>>> >> }
>>> >> return 0;
>>> >> }).forEach(System.out::println);
>>> >>
>>> >>
>>> >> // 3 But running it from the client side it fails!
>>> >> try(QueryCursor<Long> cursor = cache.query(new ScanQuery<>(), (a) ->
>>> >> 345L)){
>>> >> for( Long row : cursor) {
>>> >> System.out.println(row);
>>> >> break;
>>> >> }
>>> >> }
>>> >>
>>> >> }
>>> >>
>>> >> Also changing the Lambda to a actually class implementing
>>> IgniteClosure
>>> >> results in the same problem.
>>> >>
>>> >> Any idea what I'm doing wrong?
>>> >>
>>> >> We are running ignite ver. 2.8.0#20200226-sha1:341b01df
>>> >> With java openjdk 11.0.7 2020-04-14
>>> >>
>>> >> Thank you!
>>> >>
>>> >> Best,
>>> >> Rafael
>>> >>
>>> >>
>>> >> Error Log:
>>> >> Exception in thread "main" javax.cache.CacheException: class
>>> >> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
>>> >> [typeName=java.lang.invoke.SerializedLambda]
>>> >> at
>>> >>
>>> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.processors.cache.query.GridCacheQueryFutureAdapter.next(GridCacheQueryFutureAdapter.java:173)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$5.onHasNext(GridCacheDistributedQueryManager.java:645)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.processors.cache.AutoClosableCursorIterator.hasNext(AutoClosableCursorIterator.java:49)
>>> >> at ScanQueryTest.main(ScanQueryTest.java:49)
>>> >> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
>>> >> deserialize object [typeName=java.lang.invoke.SerializedLambda]
>>> >> at
>>> >>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10310)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.processors.cache.query.GridCacheQueryRequest.finishUnmarshal(GridCacheQueryRequest.java:389)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1625)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1843)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1468)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1365)
>>> >> at
>>> >>
>>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>>> >> at
>>> >>
>>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>>> >> at java.base/java.lang.Thread.run(Thread.java:834)
>>> >> Caused by: class org.apache.ignite.binary.BinaryObjectException:
>>> Failed to
>>> >> deserialize object [typeName=java.lang.invoke.SerializedLambda]
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:913)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1764)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:307)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:101)
>>> >> at
>>> >>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:81)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10304)
>>> >> ... 14 more
>>> >> Caused by: class org.apache.ignite.binary.BinaryObjectException:
>>> Failed to
>>> >> read field [name=capturingClass]
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
>>> >> ... 20 more
>>> >> Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:
>>> >> ScanQueryTest
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:724)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:721)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1638)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1575)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadClass(BinaryUtils.java:1552)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.readClass(BinaryReaderExImpl.java:383)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:907)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
>>> >> ... 21 more
>>> >> Caused by: java.lang.ClassNotFoundException: ScanQueryTest
>>> >> at
>>> >>
>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>>> >> at
>>> >>
>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>>> >> at
>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
>>> >> at java.base/java.lang.Class.forName0(Native Method)
>>> >> at java.base/java.lang.Class.forName(Class.java:398)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:374)
>>> >> at
>>> >>
>>> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:715)
>>> >> ... 29 more
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Rafael Troilo
>>> >> HeiGIT gGmbH
>>> >> Heidelberg Institute for Geoinformation Technology at Heidelberg
>>> University
>>> >>
>>> >> https://heigit.org | rafael.tro...@heigit.org | phone +49-6221-533
>>> 484
>>> >>
>>> >> Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg |
>>> Germany
>>> <https://www.google.com/maps/search/Schloss-Wolfsbrunnenweg+33+%7C+69118+Heidelberg+%7C+Germany?entry=gmail&source=g>
>>> >> Offices: Berliner Str. 45 | 69120 Heidelberg | Germany
>>> <https://www.google.com/maps/search/Berliner+Str.+45+%7C+69120+Heidelberg+%7C+Germany?entry=gmail&source=g>
>>> >>
>>> >> Amtsgericht Mannheim | HRB 733765
>>> >> Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger
>>> >>
>>> >
>>>
>>> --
>>> Rafael Troilo
>>> HeiGIT gGmbH
>>> Heidelberg Institute for Geoinformation Technology at Heidelberg
>>> University
>>>
>>> https://heigit.org | rafael.tro...@heigit.org | phone +49-6221-533 484
>>>
>>> Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg | Germany
>>> <https://www.google.com/maps/search/Schloss-Wolfsbrunnenweg+33+%7C+69118+Heidelberg+%7C+Germany?entry=gmail&source=g>
>>> Offices: Berliner Str. 45 | 69120 Heidelberg | Germany
>>> <https://www.google.com/maps/search/Berliner+Str.+45+%7C+69120+Heidelberg+%7C+Germany?entry=gmail&source=g>
>>>
>>> Amtsgericht Mannheim | HRB 733765
>>> Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger
>>>
>>
>
> --
> -
> Denis
>
>
