Hi everyone, I posted the following message on the Ignite Users list, but stephendarlington <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=user_nodes&user=1881> suggested that it would be better to post it in the dev list.
Original message: After the new release (2.7.0), I have been playing around with the machine learning algorithms a bit. We have some data in a cache created with the "withKeepBinary()" option, and I wanted to test if the machine learning algos would work with such a cache. I tried, but it fails with the following stacktrace: org.apache.ignite.IgniteException: testType at org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.execute(GridClosureProcessor.java:1858) at org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:568) at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6816) at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:562) at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:491) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.ignite.binary.BinaryInvalidTypeException: testType at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:707) at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1757) at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716) at org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:798) at org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:143) at org.apache.ignite.internal.processors.cache.CacheObjectUtils.unwrapBinary(CacheObjectUtils.java:177) at org.apache.ignite.internal.processors.cache.CacheObjectUtils.unwrapBinaryIfNeeded(CacheObjectUtils.java:39) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$ScanQueryIterator.advance(GridCacheQueryManager.java:3063) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$ScanQueryIterator.onHasNext(GridCacheQueryManager.java:2965) at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53) at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) at org.apache.ignite.ml.dataset.impl.cache.util.ComputeUtils.computeCount(ComputeUtils.java:313) at org.apache.ignite.ml.dataset.impl.cache.util.ComputeUtils.computeCount(ComputeUtils.java:300) at org.apache.ignite.ml.dataset.impl.cache.util.ComputeUtils.lambda$initContext$9b68d858$1(ComputeUtils.java:222) at org.apache.ignite.ml.dataset.impl.cache.util.ComputeUtils.lambda$affinityCallWithRetries$b46c4136$1(ComputeUtils.java:90) at org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.execute(GridClosureProcessor.java:1855) ... 8 common frames omitted Caused by: java.lang.ClassNotFoundException: testType at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8771) at org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:349) at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:698) ... 23 common frames omitted Debugging, I found the source of the error, at some point you are just taking the name of the upstreamCache (where the data resides), and creating a new IgniteCache object using such name before copying the data to a dataset cache. However, you are not using the keepBinary property of the original cache. I hardcoded the "withKeepBinary()" to the following lines: https://github.com/apache/ignite/blob/2.7.0/modules/ml/src/main/java/org/apache/ignite/ml/dataset/impl/cache/util/ComputeUtils.java#L162 https://github.com/apache/ignite/blob/2.7.0/modules/ml/src/main/java/org/apache/ignite/ml/dataset/impl/cache/util/ComputeUtils.java#L215 https://github.com/apache/ignite/blob/2.7.0/modules/ml/src/main/java/org/apache/ignite/ml/dataset/impl/cache/CacheBasedDatasetBuilder.java#L99 The previous made it work. I tried to retrieve the keep binary property from the upstreamCache, but I was not able to find the right method to obtain it (I saw the property is stored in the operation context field (opCtx), but it is private and cannot be accessed from the lines I modified) My example code is available at: https://gist.github.com/otorreno/ca6c5347c1bbde2d4fedd02b51d02cbb Any plans on making the machine learning algorithms work with caches with keepBinary set to true? -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/