Re: [ML] Machine Learning Pipeline Improvement

2019-05-20 Thread Alexey Zinoviev 
Dear Manu
it could be a great idea!

Could you please provide any examples of Apache Arrow integration for speed
up ML computation in another ML frameworks, it would be very helpful!

Sincerely yours
  Alexey Zinovyev



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [ML] Machine Learning Pipeline Improvement

2019-04-30 Thread Manu 
Hi, all!

Could be viable to integrate Apache Arrow to improve ML computation using
GPU?
Out of this thread, could be viable to integrate Apache Arrow to improve
Indexing computation using GPU?

Regards

https://rapids.ai   
https://arrow.apache.org   



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [ML] Machine Learning Pipeline Improvement

2018-09-07 Thread Alexey Zinoviev 
The prototype of the API will look like that

PipelineMdl mdl = new Pipeline ()
   .addFeatureExtractor(featureExtractor)
   .addLabelExtractor(lbExtractor)
   .addStage(new EncoderTrainer()
   .withEncoderType(EncoderType.STRING_ENCODER)
   .withEncodedFeature(1)
   .withEncodedFeature(6))
   .addStage(new ImputerTrainer())
   .addStage(new MinMaxScalerTrainer())
   .addStage(new NormalizationTrainer()
   .withP(1))
   .addFinalStage(new
DecisionTreeClassificationTrainer(5, 0))
   .fit(ignite, dataCache);

Also, I've added separate ticket for the update of ParamGrid/CrossValidation
API to support tune hyperparameters not only in final trainers but in
intermideate preprocessing stages too.

https://issues.apache.org/jira/browse/IGNITE-9497

I suggest to add this feature in 2.8 because it doesn't change the current
API of algorithms and has no serialized issues



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [ML] Machine Learning Pipeline Improvement

2018-08-01 Thread Yury Babak 
Sure, https://issues.apache.org/jira/browse/IGNITE-9158.

Regards,
Yury



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [ML] Machine Learning Pipeline Improvement

2018-07-30 Thread Alexey Zinoviev 
Could you please create a ticket for this task?

2018-07-20 16:47 GMT+06:00 Yury Babak :

> Alexey,
>
> I like this idea, this should improve usability of our ML module.
>
> Regards,
> Yury
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

Re: [ML] Machine Learning Pipeline Improvement

2018-07-20 Thread Yury Babak 
Alexey,

I like this idea, this should improve usability of our ML module.

Regards,
Yury



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [ML] Machine Learning Pipeline Improvement

2018-07-20 Thread Alexey Zinoviev 
Yes, it make the prerocessing easy and clear for reading and understanding.

In API it will looks like

Model mdl = Pipeline.of(reading, featureExctracting, labelExtracting,
normalizing, encoding, scaling, logisticRegression)

where in .of(...) we can see the sequence of ML stages.

Re: [ML] Machine Learning Pipeline Improvement

2018-07-19 Thread Denis Magda 
Hi Alexey,

I can't name myself an ML expert but heard that our ML component is missing
some essential data preprocessing APIs.

Are these pipelines part of our intention to bring in the preprocessing
APIs to Ignite?

--
Denis

On Thu, Jul 19, 2018 at 5:29 AM Alexey Zinoviev 
wrote:

>  Hi Igniters,
>
> I suggest to add and implement by myself sequential pipeline of machine
> learning operations including all preprocessing stages like Pipeline object
> in Python library scikit-learn (look here
>
> http://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html
> for the details)
>
> It can be combined with current Cross-Validator and Evaluator objects.
>
> The possible solution will sequentially apply a list of transforms and a
> final estimator.
>
> Alexey
>

[ML] Machine Learning Pipeline Improvement

2018-07-19 Thread Alexey Zinoviev 
 Hi Igniters,

I suggest to add and implement by myself sequential pipeline of machine
learning operations including all preprocessing stages like Pipeline object
in Python library scikit-learn (look here
http://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html
for the details)

It can be combined with current Cross-Validator and Evaluator objects.

The possible solution will sequentially apply a list of transforms and a
final estimator.

Alexey