[jira] [Created] (IGNITE-12161) [ML] Add support of different classloaders for client code

2019-09-11 Thread Alexey Platonov (Jira)
Alexey Platonov created IGNITE-12161:


 Summary: [ML] Add support of different classloaders for client code
 Key: IGNITE-12161
 URL: https://issues.apache.org/jira/browse/IGNITE-12161
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


Currently we support just one classloader for unknown classes on server side. 
Potentially client can handle several classloaders and we should support this 
situation.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Created] (IGNITE-12157) [ML] Implement distributed ROC AUC computation

2019-09-10 Thread Alexey Platonov (Jira)
Alexey Platonov created IGNITE-12157:


 Summary: [ML] Implement distributed ROC AUC computation
 Key: IGNITE-12157
 URL: https://issues.apache.org/jira/browse/IGNITE-12157
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


Currently, we don't have valid ROC AUC computation at all. We should implement 
it with predict proba interface in models. It is desirable that ROC AUC 
computation will be distributed.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Created] (IGNITE-12156) [ML] Add meta information to models

2019-09-10 Thread Alexey Platonov (Jira)
Alexey Platonov created IGNITE-12156:


 Summary: [ML] Add meta information to models
 Key: IGNITE-12156
 URL: https://issues.apache.org/jira/browse/IGNITE-12156
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


Current models don't contain any information about their learning process or a 
features meta-information or information about the type of model (for example 
RegressionTree doesn't differ from ClassificationTree in the interface). It 
leads to extra work in other APIs. For example, we cannot define automatically 
set of metrics for a user-passed model in Evaluator.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Created] (IGNITE-12155) [ML] Implementation of distributed estimator

2019-09-10 Thread Alexey Platonov (Jira)
Alexey Platonov created IGNITE-12155:


 Summary: [ML] Implementation of distributed estimator
 Key: IGNITE-12155
 URL: https://issues.apache.org/jira/browse/IGNITE-12155
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


Currently, we don't have an ability to compute metrics in a distributed manner 
and it leads to reading all data from partitions to a client using Qursor. We 
should develop a framework for distributed computing of metrics.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Created] (IGNITE-12024) [ML] PeerClassloading for ml related lambdas

2019-07-29 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-12024:


 Summary: [ML] PeerClassloading for ml related lambdas
 Key: IGNITE-12024
 URL: https://issues.apache.org/jira/browse/IGNITE-12024
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


The current implementation of peerClassloading doesn't fulfill all ml 
requirements. We want to provide ability of implementation custom vectorizers 
and preprocessors.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Created] (IGNITE-11675) [ML] Create additional examples for linear regressions, knn and kmeans

2019-04-03 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11675:


 Summary: [ML] Create additional examples for linear regressions, 
knn and kmeans
 Key: IGNITE-11675
 URL: https://issues.apache.org/jira/browse/IGNITE-11675
 Project: Ignite
  Issue Type: Improvement
  Components: documentation, ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


We should create additional examples for linear regressions, knn and kmeans in 
examples module for ebook about ML in Ignite.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11664) [ML] Use Double.NaN as default values for missing values in Vector

2019-04-01 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11664:


 Summary: [ML] Use Double.NaN as default values for missing values 
in Vector
 Key: IGNITE-11664
 URL: https://issues.apache.org/jira/browse/IGNITE-11664
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov


Currently, we use 0.0 value for default values in vectors if a value is 
missing. But this way contradicts to preprocessors politics where for missing 
values Double.NaN is using. Moreover, Double.NaN is a more convenient value for 
missing feature values.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11647) [ML] ML Vectors should work with all Serializable objects besides double

2019-03-28 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11647:


 Summary: [ML] ML Vectors should work with all Serializable objects 
besides double
 Key: IGNITE-11647
 URL: https://issues.apache.org/jira/browse/IGNITE-11647
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


Such refactoring allows us to reduce hierarchy parallelism between 
preprocessors and dataset trainers. Also, support of types besides double 
allows improving some algorithms like decision trees.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11642) [ML] Umbrella: API for Feature/Label extracting (part 2)

2019-03-27 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11642:


 Summary: [ML] Umbrella: API for Feature/Label extracting (part 2)
 Key: IGNITE-11642
 URL: https://issues.apache.org/jira/browse/IGNITE-11642
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov
 Fix For: 2.8


Replace current lambdas with fixed API



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11582) [ML] Pipelines should work with Vectorizers

2019-03-20 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11582:


 Summary: [ML] Pipelines should work with Vectorizers
 Key: IGNITE-11582
 URL: https://issues.apache.org/jira/browse/IGNITE-11582
 Project: Ignite
  Issue Type: Bug
  Components: ml
Reporter: Alexey Platonov


Currently Pipelines are implemented using feature/label extraction functions 
with generic label value. We should adapt pipelines for vectorizers (maybe 
after this ticket - IGNITE-11481).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11580) [ML] Evaluators should accept Vectorizers

2019-03-20 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11580:


 Summary: [ML] Evaluators should accept Vectorizers
 Key: IGNITE-11580
 URL: https://issues.apache.org/jira/browse/IGNITE-11580
 Project: Ignite
  Issue Type: Bug
  Components: ml
Reporter: Alexey Platonov


Currently evaluation API uses old interface with separated feature-label 
extractors. In context of IGNITE-11449 we should change this API.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11581) [ML] Adapt tutorial to new vectorizer API

2019-03-20 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11581:


 Summary: [ML] Adapt tutorial to new vectorizer API
 Key: IGNITE-11581
 URL: https://issues.apache.org/jira/browse/IGNITE-11581
 Project: Ignite
  Issue Type: Bug
  Components: ml
Reporter: Alexey Platonov


Currently tutorial uses old feature-labels extraction API



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11561) [ML] IgniteDistributedModel for XGBoost doesn't work in example

2019-03-18 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11561:


 Summary: [ML] IgniteDistributedModel for XGBoost doesn't work in 
example
 Key: IGNITE-11561
 URL: https://issues.apache.org/jira/browse/IGNITE-11561
 Project: Ignite
  Issue Type: Bug
  Components: ml
Affects Versions: 2.7, 2.8
Reporter: Alexey Platonov
Assignee: Anton Dmitriev


Distributed inference model for XGBoost doesn't work in example 
(XGBoostModelParserExample). It always returns same value.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11504) [ML] Preprocessor trainers should support new feature-label extraction API

2019-03-07 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11504:


 Summary: [ML] Preprocessor trainers should support new 
feature-label extraction API
 Key: IGNITE-11504
 URL: https://issues.apache.org/jira/browse/IGNITE-11504
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov


Problem is same as feature extractors serialization bug. We should narrow our 
API. (see parent task)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11481) [ML] Prototype of DatasetRow for Vectorizer

2019-03-05 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11481:


 Summary: [ML] Prototype of DatasetRow for Vectorizer
 Key: IGNITE-11481
 URL: https://issues.apache.org/jira/browse/IGNITE-11481
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


Vectorizer shold produce DatasetRow object that can contains columns with 
different types (double, string, etc.). It needs for preprocessors working with 
non-double values.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11480) [ML] Use only Vectorizer API in DatasetTrainer API

2019-03-05 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11480:


 Summary: [ML] Use only Vectorizer API in DatasetTrainer API  
 Key: IGNITE-11480
 URL: https://issues.apache.org/jira/browse/IGNITE-11480
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


Use only Vectorizer API in DatasetTrainer API to avoid problems with user 
classes serialization.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11479) [ML] Use new vectorizer API in PartitionDatasetBuilders

2019-03-05 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11479:


 Summary: [ML] Use new vectorizer API in PartitionDatasetBuilders
 Key: IGNITE-11479
 URL: https://issues.apache.org/jira/browse/IGNITE-11479
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


We need to exclude current feature extractors from partition building API and 
replace old extractors with new vectorizer.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11478) [ML] Use new vectorizer API in Trainers

2019-03-05 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11478:


 Summary: [ML] Use new vectorizer API in Trainers
 Key: IGNITE-11478
 URL: https://issues.apache.org/jira/browse/IGNITE-11478
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


We should rewrite current trainers - exclude all "free"-feature/labels 
extractors from APIs and use new vectorizer in them.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11476) [ML] Use new feature extraction API in examples

2019-03-05 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11476:


 Summary: [ML] Use new feature extraction API in examples
 Key: IGNITE-11476
 URL: https://issues.apache.org/jira/browse/IGNITE-11476
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


Introduce new feature/label extraction API to all examples. These examples 
should work on binary builds without sharing additional jars to libs directory 
(except ml-jar).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11477) [ML] Create tests for ML algorithms stability check against binary builds

2019-03-05 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11477:


 Summary: [ML] Create tests for ML algorithms stability check 
against binary builds 
 Key: IGNITE-11477
 URL: https://issues.apache.org/jira/browse/IGNITE-11477
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


After new feature API creation we should create tests for ML algorithms 
stability check against binary builds (or on other JVMs without common 
classpath). All new algorithms should be delivered with such test.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11475) [ML] Vectorizer API prototype with POC

2019-03-05 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11475:


 Summary: [ML] Vectorizer API prototype with POC 
 Key: IGNITE-11475
 URL: https://issues.apache.org/jira/browse/IGNITE-11475
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


We need to create a prototype of API for features/labels extraction and 
introduce it to one or two already existing examples. This prototype should 
show that new API works on binary builds.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11401) [ML] Labmdas doesn't work in binary builds

2019-02-24 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11401:


 Summary: [ML] Labmdas doesn't work in binary builds
 Key: IGNITE-11401
 URL: https://issues.apache.org/jira/browse/IGNITE-11401
 Project: Ignite
  Issue Type: Bug
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov
 Fix For: 2.8


Current lambdas for feature extraction doesn't work in binary builds and fail 
with such errors:

Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to 
deserialize object 
[typeName=org.apache.ignite.ml.regressions.linear.FeatureExtractorWrapper]
    at 
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:913)
    at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1763)
    at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1715)
    at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.readField(BinaryReaderExImpl.java:1983)
    at 
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
    at 
org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
    ... 40 more
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to read 
field [name=featureExtractor]
    at 
org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
    at 
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
    ... 45 more
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to 
deserialize object [typeName=java.lang.invoke.SerializedLambda]
    at 
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:913)
    at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1763)
    at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1715)
    at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.readField(BinaryReaderExImpl.java:1983)
    at 
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
    at 
org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
    ... 46 more
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to read 
field [name=capturedArgs]
    at 
org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
    at 
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
    ... 51 more
Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException: 
org.apache.ignite.examples.ml.regression.linear.StaticFun

 





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11372) [ML] Fix javadoc in VectorWithDistributionId

2019-02-20 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11372:


 Summary: [ML] Fix javadoc in VectorWithDistributionId
 Key: IGNITE-11372
 URL: https://issues.apache.org/jira/browse/IGNITE-11372
 Project: Ignite
  Issue Type: Bug
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


"mvn initialize -Pjavadoc" fails with error

"[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-antrun-plugin:1.7:run 
(javadoc-postprocessing-new) on project apache-ignite: An Ant BuildException 
has occured: Execution failed due to: Class doesn't have description in file: 
<..>/ml/util/generators/primitives/vector/VectorGeneratorsFamily.VectorWithDistributionId.html"



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11241) [ML] blas NoClassDefFoundError exception

2019-02-07 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11241:


 Summary: [ML] blas NoClassDefFoundError exception
 Key: IGNITE-11241
 URL: https://issues.apache.org/jira/browse/IGNITE-11241
 Project: Ignite
  Issue Type: Bug
  Components: ml
Affects Versions: 2.7, 2.6, 2.5, 2.4, 2.3, 2.2, 2.1, 2.0, 2.8
Reporter: Alexey Platonov
Assignee: Alexey Platonov
 Fix For: 2.8


Caused by: java.lang.NoClassDefFoundError: org/netlib/blas/Dnrm2 
        at com.github.fommil.netlib.F2jBLAS.dnrm2(F2jBLAS.java:121) 
        at 
org.apache.ignite.ml.math.isolve.lsqr.LSQROnHeap.lambda$bnorm$7b55d622$1(LSQROnHeap.java:53)
 
        at 
org.apache.ignite.ml.dataset.Dataset.lambda$computeWithCtx$96d91738$1(Dataset.java:143)
 
        at 
org.apache.ignite.ml.dataset.impl.cache.CacheBasedDataset.lambda$computeWithCtx$62946a87$1(CacheBasedDataset.java:112)
 
        at 
org.apache.ignite.ml.dataset.impl.cache.util.ComputeUtils.lambda$affinityCallWithRetries$b46c4136$1(ComputeUtils.java:90)
 
        at 
org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.execute(GridClosureProcessor.java:1855)
 
        at 
org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:568)
 
        at 
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6816)
 
        at 
org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:562)
 
        ... 11 more



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11234) [ML] Add fillCache method to DataStreamGenerator

2019-02-06 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11234:


 Summary: [ML] Add fillCache method to DataStreamGenerator
 Key: IGNITE-11234
 URL: https://issues.apache.org/jira/browse/IGNITE-11234
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


We need to add fillCache(sampleSize, cache) method to DataStreamGenerator for a 
more convenient interface in terms of cache filling + add example for such 
method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11232) [ML] Random Forest, NodeId exception (Failed to serialize object)

2019-02-06 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11232:


 Summary: [ML] Random Forest, NodeId exception (Failed to serialize 
object)
 Key: IGNITE-11232
 URL: https://issues.apache.org/jira/browse/IGNITE-11232
 Project: Ignite
  Issue Type: Bug
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
 serialize object: IgniteBiTuple [val1=0, val2=1]
 at
 
org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.marshal0(OptimizedMarshaller.java:207)
 at
 
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
 at
 org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10222)
 at
 
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:196)
 ... 99 more
 Caused by: java.io.IOException: Externalizable class doesn't have default
 constructor: class org.apache.ignite.ml.tree.randomforest.data.NodeId
 at
 
org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.(OptimizedClassDescriptor.java:408)
 at
 
org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.classDescriptor(OptimizedMarshallerUtils.java:210)
 at
 
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectOutputStream.writeObject0(OptimizedObjectOutputStream.java:201)
 at
 
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectOutputStream.writeObjectOverride(OptimizedObjectOutputStream.java:159)
 at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
 at
 
org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.marshal0(OptimizedMarshaller.java:202)
 ... 102 more
 Caused by: java.lang.NoSuchMethodException:
 org.apache.ignite.ml.tree.randomforest.data.NodeId.()
 at java.lang.Class.getConstructor0(Class.java:3082)
 at java.lang.Class.getDeclaredConstructor(Class.java:2178)
 at
 
org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.(OptimizedClassDescriptor.java:403)
 ... 107 more



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11192) [ML] Use nd4j for matrix inversions and determinants

2019-02-04 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11192:


 Summary: [ML] Use nd4j for matrix inversions and determinants
 Key: IGNITE-11192
 URL: https://issues.apache.org/jira/browse/IGNITE-11192
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov


>From optimization point of view we should use matrix inversions and 
>determinant computations of dl4j instead of own realization.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11133) Refactoring of compatibility framework

2019-01-30 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-11133:


 Summary: Refactoring of compatibility framework
 Key: IGNITE-11133
 URL: https://issues.apache.org/jira/browse/IGNITE-11133
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Platonov
Assignee: Alexey Platonov


We need to refactor current compatibility framework with the aim of reducing 
maven management logic, simplification adding new versions to check.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-10793) [ML] Create comprehensive example for dataset generators

2018-12-21 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-10793:


 Summary: [ML] Create comprehensive example for dataset generators 
 Key: IGNITE-10793
 URL: https://issues.apache.org/jira/browse/IGNITE-10793
 Project: Ignite
  Issue Type: Sub-task
  Components: ml
Reporter: Alexey Platonov
Assignee: Alexey Platonov
 Fix For: 2.8


We  need to create well documented example for generators.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-10727) [ML] InfModel and Model merging

2018-12-17 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-10727:


 Summary: [ML] InfModel and Model merging 
 Key: IGNITE-10727
 URL: https://issues.apache.org/jira/browse/IGNITE-10727
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov
Assignee: Anton Dmitriev
 Fix For: 2.8


Currently "InfModel" and "Model" provide parallel architecture in terms of 
using of "InfModels" in after-learning steps like compositions, estimations 
etc. I propose to move "InfModel" to top of models hierarchy and rename it to 
just "Model" and current "Model" rename to 
"LearnedModel/LocalModel/Your_ad_may_be_here". So in this way "InfModel" will 
be just model with apply-function for generic I-O values and "Model" will be 
serializable thing on Vector-Double.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-10549) [ML] LogisticRegressionSGDTrainerTest fails with BLAS

2018-12-05 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-10549:


 Summary: [ML] LogisticRegressionSGDTrainerTest fails with BLAS
 Key: IGNITE-10549
 URL: https://issues.apache.org/jira/browse/IGNITE-10549
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Platonov
Assignee: Alexey Platonov


LogisticRegressionSGDTrainerTest fails with BLAS on case with 3 partitions. 
Investigation is needed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-10517) [ML] Merge inference and learning architectures

2018-12-04 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-10517:


 Summary: [ML] Merge inference and learning architectures
 Key: IGNITE-10517
 URL: https://issues.apache.org/jira/browse/IGNITE-10517
 Project: Ignite
  Issue Type: Sub-task
  Components: ml
Reporter: Alexey Platonov
 Fix For: 2.8


We need to review of inference framework and try to merge it with core part on 
ML library in terms of reusing imported models in all ML steps excluding 
learning (model estimation, compositions etc.)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-10503) Meta information for vectors

2018-12-02 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-10503:


 Summary: Meta information for vectors
 Key: IGNITE-10503
 URL: https://issues.apache.org/jira/browse/IGNITE-10503
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Alexey Platonov


We need to design and implement vector meta-information like feature names, 
bagging information, etc



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-10440) Analyse test suites for possible acceleration

2018-11-28 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-10440:


 Summary: Analyse test suites for possible acceleration
 Key: IGNITE-10440
 URL: https://issues.apache.org/jira/browse/IGNITE-10440
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Platonov
Assignee: Alexey Platonov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-10195) Cannot create caches with different names but with same indexed types and schema name

2018-11-08 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-10195:


 Summary: Cannot create caches with different names but with same 
indexed types and schema name
 Key: IGNITE-10195
 URL: https://issues.apache.org/jira/browse/IGNITE-10195
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Platonov


Cannot create caches with different names but with same indexed types and 
schema name. For example, such code will throw exception 
"javax.cache.CacheException: Table already exists: VALUE".

 

{color:#9876aa}node{color}.createCache({color:#cc7832}new 
{color}CacheConfiguration()
 .setName({color:#9876aa}"PERSON_1"{color})
 .setIndexedTypes(Key.{color:#cc7832}class,{color} 
Person.{color:#cc7832}class{color})
 .setSqlSchema(QueryUtils.{color:#9876aa}DFLT_SCHEMA{color})){color:#cc7832};
{color}{color:#cc7832}
{color}{color:#9876aa}node{color}.createCache({color:#cc7832}new 
{color}CacheConfiguration()
 .setName({color:#9876aa}"PERSON_2"{color})
 .setIndexedTypes({color:#cc7832}Key.class, Person.class{color})
 
.setSqlSchema(QueryUtils.{color:#9876aa}DFLT_SCHEMA{color})){color:#cc7832};{color}

 

If I set table name manually by setQueryEntities(...) then 
"javax.cache.CacheException: Index already exists: PERSON_ORGID_IDX" wil be 
thrown (Value has field with "origId" and annotation 
{color:#bbb529}@QuerySqlField{color}({color:#d0d0ff}index {color}= 
{color:#cc7832}true{color})). Here is definition of Person class:

 

{color:#cc7832}public static class {color}PersonKey {
 {color:#bbb529}@QuerySqlField
{color} {color:#cc7832}public long 
{color}{color:#9876aa}id{color}{color:#cc7832};
{color}{color:#cc7832}
{color} {color:#629755}/**
{color}{color:#629755} * Constructor.
{color}{color:#629755} *
{color}{color:#629755} * {color}{color:#629755}@param {color}{color:#8a653b}id 
{color}{color:#629755}ID.
{color}{color:#629755} */
{color} {color:#ffc66d}PersonKey{color}({color:#cc7832}long {color}id) {
 {color:#cc7832}this{color}.{color:#9876aa}id {color}= id{color:#cc7832};
{color} }

 {color:#629755}/** {{color}{color:#629755}@inheritDoc{color}{color:#629755}} */
{color} {color:#bbb529}@Override {color}{color:#cc7832}public int 
{color}{color:#ffc66d}hashCode{color}() {
 {color:#cc7832}return 
{color}({color:#cc7832}int{color}){color:#9876aa}id{color}{color:#cc7832};
{color} }

 {color:#629755}/** {{color}{color:#629755}@inheritDoc{color}{color:#629755}} */
{color} {color:#bbb529}@Override {color}{color:#cc7832}public boolean 
{color}{color:#ffc66d}equals{color}(Object obj) {
 {color:#cc7832}return {color}obj != {color:#cc7832}null {color}&& obj 
{color:#cc7832}instanceof {color}PersonKey && 
(F.eq({color:#9876aa}id{color}{color:#cc7832}, 
{color}((PersonKey)obj).{color:#9876aa}id{color})){color:#cc7832};
{color} }
}

 

Such behavior seems to be usability bug. Why I cannot create two caches with 
different names but with same indexed values.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-10194) ZookeeperDiscoverySpiTest fails on testLocalAuthenticationFails

2018-11-08 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-10194:


 Summary: ZookeeperDiscoverySpiTest fails on 
testLocalAuthenticationFails
 Key: IGNITE-10194
 URL: https://issues.apache.org/jira/browse/IGNITE-10194
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Platonov
Assignee: Alexey Platonov


We need to unmute test (remove "_" prefix) and fix this test



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-10141) Cache 3 tests optimization

2018-11-06 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-10141:


 Summary: Cache 3 tests optimization
 Key: IGNITE-10141
 URL: https://issues.apache.org/jira/browse/IGNITE-10141
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Platonov
Assignee: Alexey Platonov


We need to investigate how to optimize these tests:

 

GridCacheInterceptorTransactionalRebalanceTest.testRebalanceUpdate
GridCacheInterceptorTransactionalRebalanceTest.testRebalanceRemoveInvoke
GridCacheInterceptorTransactionalRebalanceTest.testPutIfAbsent
GridCacheInterceptorTransactionalRebalanceTest.testRebalanceUpdateInvoke
IgniteCacheGroupsTest.testRestartsAndCacheCreateDestroy
GridCacheInterceptorTransactionalRebalanceTest.testRebalanceRemove
GridCacheInterceptorTransactionalRebalanceTest.testGetAndPut
IgniteCacheGroupsTest.testStartManyCaches
GridCacheVersionTopologyChangeTest.testVersionIncreaseTx
GridCacheVersionTopologyChangeTest.testVersionIncreaseAtomic
GridCacheValueConsistencyTransactionalSelfTest.testPutRemoveConsistencyMultithreaded
GridCacheInterceptorAtomicRebalanceTest.testRebalanceUpdateInvoke
GridCacheInterceptorAtomicRebalanceTest.testGetAndPut

 

and optimize them.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-10106) Cache 5 test suite optimization

2018-11-01 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-10106:


 Summary: Cache 5 test suite optimization
 Key: IGNITE-10106
 URL: https://issues.apache.org/jira/browse/IGNITE-10106
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Platonov
Assignee: Alexey Platonov


We need to investigate how to optimize these tests:
|CacheSerializableTransactionsTest.testGetRemoveTxNearCache2|
|[CacheSerializableTransactionsTest.testGetRemoveTxNearCache1|https://ci.ignite.apache.org/viewLog.html?currentGroup=test&scope=org.apache.ignite.testsuites.IgniteCacheTestSuite5%23teamcity%23org.apache.ignite.internal.processors.cache%23teamcity%23CacheSerializableTransactionsTest&pager.currentPage=1&order=DURATION_DESC&recordsPerPage=20&filterText=&status=&buildTypeId=IgniteTests24Java8_Cache5&buildId=2160072&tab=testsInfo]|
|CacheSerializableTransactionsTest.testRandomOperations|
|[CacheSerializableTransactionsTest.testIncrementTxRestart|https://ci.ignite.apache.org/viewLog.html?currentGroup=test&scope=org.apache.ignite.testsuites.IgniteCacheTestSuite5%23teamcity%23org.apache.ignite.internal.processors.cache%23teamcity%23CacheSerializableTransactionsTest&pager.currentPage=1&order=DURATION_DESC&recordsPerPage=20&filterText=&status=&buildTypeId=IgniteTests24Java8_Cache5&buildId=2160072&tab=testsInfo]|
|CacheSerializableTransactionsTest.testConcurrentUpdateNoDeadlockNodeRestart|
|[CacheSerializableTransactionsTest.testConcurrentUpdateNoDeadlockFromClientsNodeRestart|https://ci.ignite.apache.org/viewLog.html?currentGroup=test&scope=org.apache.ignite.testsuites.IgniteCacheTestSuite5%23teamcity%23org.apache.ignite.internal.processors.cache%23teamcity%23CacheSerializableTransactionsTest&pager.currentPage=1&order=DURATION_DESC&recordsPerPage=20&filterText=&status=&buildTypeId=IgniteTests24Java8_Cache5&buildId=2160072&tab=testsInfo]|
|[CacheSerializableTransactionsTest.testConcurrentUpdateNoDeadlockWithNonSerializable|https://ci.ignite.apache.org/viewLog.html?currentGroup=test&scope=org.apache.ignite.testsuites.IgniteCacheTestSuite5%23teamcity%23org.apache.ignite.internal.processors.cache%23teamcity%23CacheSerializableTransactionsTest&pager.currentPage=1&order=DURATION_DESC&recordsPerPage=20&filterText=&status=&buildTypeId=IgniteTests24Java8_Cache5&buildId=2160072&tab=testsInfo]|
|[CacheSerializableTransactionsTest.testConcurrentUpdateNoDeadlockGetPut|https://ci.ignite.apache.org/viewLog.html?currentGroup=test&scope=org.apache.ignite.testsuites.IgniteCacheTestSuite5%23teamcity%23org.apache.ignite.internal.processors.cache%23teamcity%23CacheSerializableTransactionsTest&pager.currentPage=1&order=DURATION_DESC&recordsPerPage=20&filterText=&status=&buildTypeId=IgniteTests24Java8_Cache5&buildId=2160072&tab=testsInfo]|
|[CacheSerializableTransactionsTest.testConcurrentUpdateNoDeadlockFromClients|https://ci.ignite.apache.org/viewLog.html?currentGroup=test&scope=org.apache.ignite.testsuites.IgniteCacheTestSuite5%23teamcity%23org.apache.ignite.internal.processors.cache%23teamcity%23CacheSerializableTransactionsTest&pager.currentPage=1&order=DURATION_DESC&recordsPerPage=20&filterText=&status=&buildTypeId=IgniteTests24Java8_Cache5&buildId=2160072&tab=testsInfo]|
|[CacheSerializableTransactionsTest.testConcurrentUpdateNoDeadlock|https://ci.ignite.apache.org/viewLog.html?currentGroup=test&scope=org.apache.ignite.testsuites.IgniteCacheTestSuite5%23teamcity%23org.apache.ignite.internal.processors.cache%23teamcity%23CacheSerializableTransactionsTest&pager.currentPage=1&order=DURATION_DESC&recordsPerPage=20&filterText=&status=&buildTypeId=IgniteTests24Java8_Cache5&buildId=2160072&tab=testsInfo]|

and optimize them.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-10037) Cache 2 tests optimization

2018-10-29 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-10037:


 Summary: Cache 2 tests optimization
 Key: IGNITE-10037
 URL: https://issues.apache.org/jira/browse/IGNITE-10037
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Platonov
Assignee: Alexey Platonov


We need to investigate how to optimize these tests:

GridCachePartitionNotLoadedEventSelfTest.testPrimaryAndBackupDead
 
CacheTxLoadingConcurrentGridStartSelfTestAllowOverwrite.testLoadCacheWithDataStreamerSequentialWithConfigAndRestarts
 IgniteCacheEntryProcessorNodeJoinTest.testEntryProcessorNodeLeave
 
CacheTxLoadingConcurrentGridStartSelfTestAllowOverwrite.testLoadCacheWithDataStreamerSequentialWithConfig
 
CacheTxLoadingConcurrentGridStartSelfTestAllowOverwrite.testLoadCacheWithDataStreamerSequentialClientWithConfig

 

and optimize them.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-10008) Need to investigate why these tests is failed after consistentID set

2018-10-25 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-10008:


 Summary: Need to investigate why these tests is failed after 
consistentID set
 Key: IGNITE-10008
 URL: https://issues.apache.org/jira/browse/IGNITE-10008
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Platonov


In these test method setConsistentID is overridden because setting of 
consistentID to nodes will let fail them:

JdbcDistributedJoinsQueryTest
GridCacheAbstractMetricsSelfTest
IgniteCacheContinuousQueryNoUnsubscribeTest
IgniteBinaryObjectFieldsQuerySelfTest
IgniteSqlSkipReducerOnUpdateDmlSelfTest



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9979) [ML] Implement Naive Bayes classifier over Discrete Disctribution

2018-10-23 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-9979:
---

 Summary: [ML] Implement Naive Bayes classifier over Discrete 
Disctribution
 Key: IGNITE-9979
 URL: https://issues.apache.org/jira/browse/IGNITE-9979
 Project: Ignite
  Issue Type: Sub-task
  Components: ml
Reporter: Alexey Platonov
 Fix For: 2.8


There is need to create Naive Bayes classifier over simple Discrete 
Distribution for non-continuous features in dataset. As naive implementation it 
may be set of histograms of feature values for each feature in dataset.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9978) [ML] Implement Compound Naive Bayes classifier

2018-10-23 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-9978:
---

 Summary: [ML] Implement Compound Naive Bayes classifier
 Key: IGNITE-9978
 URL: https://issues.apache.org/jira/browse/IGNITE-9978
 Project: Ignite
  Issue Type: Task
  Components: ml
Reporter: Alexey Platonov
 Fix For: 2.8


We need to create compound Naive Bayes classifier as model composition of 
several Naive Bayes classifiers where each classifier represents subset of 
features of one type. For example such model may contain Naive Bayes model over 
Gauss Distribution for all continuous features and Naive Bayes model over 
Discrete Distribution for enum-like features.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9959) Set consistent id in tests

2018-10-22 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-9959:
---

 Summary: Set consistent id in tests
 Key: IGNITE-9959
 URL: https://issues.apache.org/jira/browse/IGNITE-9959
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Platonov
Assignee: Alexey Platonov


We need to set consistent id for starting nodes in tests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9930) Split Zookeeper Discovery 2 onto several configs

2018-10-18 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-9930:
---

 Summary: Split Zookeeper Discovery 2 onto several configs
 Key: IGNITE-9930
 URL: https://issues.apache.org/jira/browse/IGNITE-9930
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Platonov
Assignee: Alexey Platonov


At the current state this config require a lot of time for one test-run 
(>100min). We need to split this onto several configs with running time <= 
60min.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9886) Split Continuous Query 1 onto several config

2018-10-15 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-9886:
---

 Summary: Split Continuous Query 1 onto several config
 Key: IGNITE-9886
 URL: https://issues.apache.org/jira/browse/IGNITE-9886
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Platonov
Assignee: Alexey Platonov


At the current state this config require a lot of time for one test-run 
(>100min). We need to split this onto several configs with running time <= 
60min.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9834) ClientImpl shouldn't fail JVM with 130 error code after node validation errors

2018-10-10 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-9834:
---

 Summary: ClientImpl shouldn't fail JVM with 130 error code after 
node validation errors
 Key: IGNITE-9834
 URL: https://issues.apache.org/jira/browse/IGNITE-9834
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Platonov
Assignee: Alexey Platonov


In the current number of Ignite if client (tcp-client-disco-msg-worker) receive 
validation error message represented by TcpDiscoveryCheckFailedMessage then 
client fail with 130 error code. Semantically such behavior is invalid. Client 
must cancel own work in normal order.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9829) Add validation error distinction to TcpDiscoveryCheckFailedMessage

2018-10-09 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-9829:
---

 Summary: Add validation error distinction to 
TcpDiscoveryCheckFailedMessage
 Key: IGNITE-9829
 URL: https://issues.apache.org/jira/browse/IGNITE-9829
 Project: Ignite
  Issue Type: Improvement
  Components: messaging
Reporter: Alexey Platonov
 Fix For: 3.0


There is no way to define validation error type during joining on client side 
after node validation on server side. TcpDiscoveryCheckFailedMessage just 
aggregates error messages to one string. For example: if there was auth error 
while joining then client receive TcpDiscoveryCheckFailedMessage instead of 
TcpDiscoveryAuthFailedMessage. This is due to server validation protocol 
supports just IgniteNodeValidationResult as string-wrapper without additional 
information about types of validation errors. Set of enum values representing 
validation errors may be good solution.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9726) GridCacheAbstractFailoverSelfTest may lock all suite on put/remove cache operations

2018-09-27 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-9726:
---

 Summary: GridCacheAbstractFailoverSelfTest may lock all suite on 
put/remove cache operations
 Key: IGNITE-9726
 URL: https://issues.apache.org/jira/browse/IGNITE-9726
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Platonov
Assignee: Alexey Platonov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9675) Deadlock on Ignite:active() and stopping grid simultaneously calling

2018-09-24 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-9675:
---

 Summary: Deadlock on Ignite:active() and stopping grid 
simultaneously calling
 Key: IGNITE-9675
 URL: https://issues.apache.org/jira/browse/IGNITE-9675
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Platonov
Assignee: Alexey Platonov


There is deadlock on Ignite:active() and stopping grid simultaneously calling
 # Trying to stop client node.

{code:java}
"main-ScalaTest-running-VisorInProcDriverSpec" #1 prio=5 os_prio=0 
tid=0x7f267800e800 nid=0x6574 sleeping[0x7f2681b7e000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at 
org.apache.ignite.internal.util.GridSpinReadWriteLock.writeLock(GridSpinReadWriteLock.java:206)
at 
org.apache.ignite.internal.processors.task.GridTaskProcessor.onKernalStop(GridTaskProcessor.java:190)
at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2135)
at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2083)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2590)
- locked <0x000797ecf7f8> (a 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2553)
at org.apache.ignite.internal.IgnitionEx.stop(IgnitionEx.java:374)
at org.apache.ignite.Ignition.stop(Ignition.java:225)
{code}
 

and
 2. Execute task that tries to get ignite.active() state, which also executes 
task 
GridClusterStateProcessor.sendComputeCheckGlobalState(GridClusterStateProcessor.java:1086)
{code:java}
"mgmt-#2470" #2965 prio=5 os_prio=0 tid=0x7f23fc001000 nid=0x730b waiting 
on condition [0x7f221f0ee000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
at 
org.apache.ignite.internal.AsyncSupportAdapter.saveOrGet(AsyncSupportAdapter.java:112)
at 
org.apache.ignite.internal.IgniteComputeImpl.call(IgniteComputeImpl.java:786)
at 
org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.sendComputeCheckGlobalState(GridClusterStateProcessor.java:1086)
at 
org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.publicApiActiveState(GridClusterStateProcessor.java:177)
at 
org.apache.ignite.internal.cluster.IgniteClusterImpl.active(IgniteClusterImpl.java:300)
at 
org.apache.ignite.internal.visor.node.VisorNodeDataCollectorTask.reduce(VisorNodeDataCollectorTask.java:75)
at 
org.gridgain.grid.internal.visor.node.VisorGridGainNodeDataCollectorTask.reduce0(VisorGridGainNodeDataCollectorTask.java:39)
at 
org.gridgain.grid.internal.visor.node.VisorGridGainNodeDataCollectorTask.reduce0(VisorGridGainNodeDataCollectorTask.java:26)
at 
org.apache.ignite.internal.visor.VisorMultiNodeTask.reduce(VisorMultiNodeTask.java:139)
at 
org.apache.ignite.internal.processors.task.GridTaskWorker$6.call(GridTaskWorker.java:1139)
at 
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6765)
at 
org.apache.ignite.internal.processors.task.GridTaskWorker.reduce(GridTaskWorker.java:1137)
at 
org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:964)
at 
org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
at 
org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
at 
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9635) Eternal initial partition map exchange

2018-09-18 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-9635:
---

 Summary: Eternal initial partition map exchange
 Key: IGNITE-9635
 URL: https://issues.apache.org/jira/browse/IGNITE-9635
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Platonov


Sometimes test suites times out on test class 
GridCacheReplicatedNodeRestartSelfTest. Finally, some test from this class 
blocked on awaiting node starting due to eternal initial partition map exchange.

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9631) Timeouts in ZookeeperDIscoverySpi test suite

2018-09-18 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-9631:
---

 Summary: Timeouts in ZookeeperDIscoverySpi test suite
 Key: IGNITE-9631
 URL: https://issues.apache.org/jira/browse/IGNITE-9631
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Platonov
Assignee: Alexey Platonov


Sometimes this test suite times out. Failed test examples:

[https://ci.ignite.apache.org/viewLog.html?buildId=1862120&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_ZooKeeperDiscovery2]

[https://ci.ignite.apache.org/viewLog.html?buildId=1839809&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_ZooKeeperDiscovery2]

It seems that test class GridCachePartitionedNodeRestartTest times out and 
block all suite.

We need to add timeouts into test for suite timeouts preventing.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9580) Fix exit code 137 in Query 1 Suite

2018-09-13 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-9580:
---

 Summary: Fix exit code 137 in Query 1 Suite
 Key: IGNITE-9580
 URL: https://issues.apache.org/jira/browse/IGNITE-9580
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Platonov
Assignee: Alexey Platonov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9531) ZookeeperDiscovery testClientReconnect is flaky in master

2018-09-11 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-9531:
---

 Summary: ZookeeperDiscovery testClientReconnect is flaky in master
 Key: IGNITE-9531
 URL: https://issues.apache.org/jira/browse/IGNITE-9531
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Platonov
Assignee: Alexey Platonov
 Fix For: 2.8


The test IgniteClientReconnectCacheTest#testReconnectMultinode(LongHistory) 
periodically fails with timeouts in master.
>From the logs I see that the hang is caused by one of the two assertion errors:
{code}
java.lang.AssertionError
at 
org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.checkClientsStatus(ZookeeperDiscoveryImpl.java:1345)
at 
org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.access$2300(ZookeeperDiscoveryImpl.java:108)
at 
org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl$CheckClientsStatusCallback.processResult0(ZookeeperDiscoveryImpl.java:4332)
at 
org.apache.ignite.spi.discovery.zk.internal.ZkAbstractChildrenCallback.processResult(ZkAbstractChildrenCallback.java:42)
at 
org.apache.ignite.spi.discovery.zk.internal.ZookeeperClient$ChildrenCallbackWrapper.processResult(ZookeeperClient.java:1132)
at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
{code}
or 
{code}
java.lang.AssertionError
at 
org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.checkClientsStatus(ZookeeperDiscoveryImpl.java:1388)
at 
org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.access$2300(ZookeeperDiscoveryImpl.java:108)
at 
org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl$CheckClientsStatusCallback.processResult0(ZookeeperDiscoveryImpl.java:4332)
at 
org.apache.ignite.spi.discovery.zk.internal.ZkAbstractChildrenCallback.processResult(ZkAbstractChildrenCallback.java:42)
at 
org.apache.ignite.spi.discovery.zk.internal.ZookeeperClient$ChildrenCallbackWrapper.processResult(ZookeeperClient.java:1132)
at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
{code}

The test failure can be rarely reproduced locally (run repeatedly with CPU 
stress enabled).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9022) [ML] Implement class labels mapping for SVM binary classifier

2018-07-17 Thread Alexey Platonov (JIRA)
Alexey Platonov created IGNITE-9022:
---

 Summary: [ML] Implement class labels mapping for SVM binary 
classifier
 Key: IGNITE-9022
 URL: https://issues.apache.org/jira/browse/IGNITE-9022
 Project: Ignite
  Issue Type: Bug
  Components: ml
Reporter: Alexey Platonov
Assignee: Aleksey Zinoviev
 Fix For: 2.7


We need to automatically compute mapping from user's labels to \{-1;1} for SVM.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)