[jira] [Commented] (IGNITE-532) Implement IgniteAkkaStreamer to stream data from Akka actors.

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073393#comment-16073393
 ] 

Anton Vinogradov commented on IGNITE-532:
-

Maxim, 

Seems we have no Scala expertise inside Ignite Project, so issue can't be 
reviewed.
So, I see 2 ways here: 
1) To rewrite code using java
2) To implement this as a part of Akka project. 

I'm changing status from PATCH AVAILABLE to IN_PROGRESS until decision not 
taken.


> Implement IgniteAkkaStreamer to stream data from Akka actors.
> -
>
> Key: IGNITE-532
> URL: https://issues.apache.org/jira/browse/IGNITE-532
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Maksim Kozlov
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Akka|http://akka.io/] for more information. Given that Akka is a Scala 
> frameworks, this streamer should be available in Scala.
> We should create {{IgniteAkkaStreamer}} which will consume messages from Akka 
> Actors and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> * Convert data from Akka to Ignite using an optional pluggable converter. If 
> not provided, then we should have some default mechanism.
> * Specify the cache name for the Ignite cache to load data into.
> * Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-532) Implement IgniteAkkaStreamer to stream data from Akka actors.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021347#comment-16021347
 ] 

ASF GitHub Bot commented on IGNITE-532:
---

GitHub user dream-x opened a pull request:

https://github.com/apache/ignite/pull/1998

IGNITE-532: Implement IgniteAkkaStreamer to stream data from Akka actors.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dream-x/ignite ignite-532

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1998.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1998


commit f47ab06b1ca4416eefc10ee80237898864dc67e5
Author: Max Kozlov 
Date:   2017-04-03T14:02:17Z

Init akka stream module.

commit d105635dbe0ae8617404ef82a933a181a07cad8e
Author: Max Kozlov 
Date:   2017-04-04T09:49:09Z

Add akka-actor and -stream dependency in pom.xml.

commit d44d4edaa07d11b9e69aef5e20f4e4ab1249d64e
Author: Max Kozlov 
Date:   2017-04-06T16:45:54Z

Update akka dependency.

commit c63fc6290558fb552fb84e1213d2f85c9e51ddc6
Author: Max Kozlov 
Date:   2017-04-06T16:46:19Z

wip

commit 4aafc72c70941c89da36a26d3d00e23f2f384246
Author: Max Kozlov 
Date:   2017-04-13T08:47:01Z

Add akka-stream streamer.

commit f9b35d691517822fee29dd61cc10b4b160bbd3f6
Author: Max Kozlov 
Date:   2017-04-13T08:47:15Z

Add akka-stream streamer test.

commit b79bc8dfaf9e7ef8a385ee245b09629216e10b17
Author: Max Kozlov 
Date:   2017-04-13T08:47:36Z

Delete ExampleAkkaStreamToIgniteStream.java

commit a835cee93c86ba5f9f5119621d2e5efe73423ed5
Author: Max Kozlov 
Date:   2017-04-13T09:01:56Z

Delete onComplete test.

commit 513d3c75be5ec625f1ca5b6b3d3b27ed2d75aab3
Author: Max Kozlov 
Date:   2017-04-13T10:45:51Z

wip.

commit 924c9ae96b0a91c662ad9c06bd7c919f4935faf6
Author: Max Kozlov 
Date:   2017-04-14T08:24:28Z

Change version akka to 2.5.

commit 09d26b9972931e5595a7bb30e9796ef8816ff455
Author: Max Kozlov 
Date:   2017-04-18T14:35:18Z

Ignite streamer based on Scala Actor.

commit cc3548f429484e24a2454ec2e98961d102e3fe55
Author: Max Kozlov 
Date:   2017-04-19T13:57:29Z

Refactoring.

commit a912475488da162df8f7662489382fec90716024
Author: Max Kozlov 
Date:   2017-04-27T14:35:59Z

wip

commit 5dde548ff359f466de9df155aacc093cbfab77fd
Author: Max Kozlov 
Date:   2017-05-02T13:32:52Z

Add test akka actor streamer.

commit cb45c197d377e13f9d489bcc562615d8d7e5a3a2
Author: Max Kozlov 
Date:   2017-05-03T11:10:36Z

Refactoring scala tests.

commit b5be16c5221a91467699aec487543c676635433b
Author: Max Kozlov 
Date:   2017-05-03T11:15:14Z

Update akka to 2.5

commit ffc3c9975e6dd8fb2e881d6a0c6b3f2708208874
Author: Max Kozlov 
Date:   2017-05-03T12:33:18Z

Update pom.xml

commit 6e9203eec0096f168c89158c16659808f14cc498
Author: Max Kozlov 
Date:   2017-05-04T13:31:22Z

Refactoring.

commit 318748ab26819eab16be9a971b26386ed0244d4c
Author: Max Kozlov 
Date:   2017-05-15T09:24:46Z

Delete unused code.

commit 805490ef138bc351ac357cc80de5329115a4a5a7
Author: Max Kozlov 
Date:   2017-05-22T14:30:45Z

Remove java classes. Refactor imports.

commit ec7917681480d3b5476a141a30e86b7c89609e4f
Author: Max Kozlov 
Date:   2017-05-23T15:36:41Z

Refactoring code.




> Implement IgniteAkkaStreamer to stream data from Akka actors.
> -
>
> Key: IGNITE-532
> URL: https://issues.apache.org/jira/browse/IGNITE-532
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Maksim Kozlov
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Akka|http://akka.io/] for more information. Given that Akka is a Scala 
> frameworks, this streamer should be available in Scala.
> We should create {{IgniteAkkaStreamer}} which will consume messages from Akka 
> Actors and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> * Convert data from Akka to Ignite using an optional pluggable converter. If 
> not provided, then we should have some default mechanism.
> * 

[jira] [Commented] (IGNITE-532) Implement IgniteAkkaStreamer to stream data from Akka actors.

[Maksim Kozlov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1596#comment-1596
 ] 

Maksim Kozlov commented on IGNITE-532:
--

Hi guys,
I have some questions about the issue:
1. Akka-stream workflow implementation few stages:
{noformat}
akka-stream source -> akka-stream flow -> akka-stream sink (ignite-stream)
{noformat}
Will it be enough to implement the creation of a Sink object in ignite-stream 
for workflow akka-stream?
2. Support for the akka-actor is relevant?

> Implement IgniteAkkaStreamer to stream data from Akka actors.
> -
>
> Key: IGNITE-532
> URL: https://issues.apache.org/jira/browse/IGNITE-532
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Maksim Kozlov
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Akka|http://akka.io/] for more information. Given that Akka is a Scala 
> frameworks, this streamer should be available in Scala.
> We should create {{IgniteAkkaStreamer}} which will consume messages from Akka 
> Actors and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> * Convert data from Akka to Ignite using an optional pluggable converter. If 
> not provided, then we should have some default mechanism.
> * Specify the cache name for the Ignite cache to load data into.
> * Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-532) Implement IgniteAkkaStreamer to stream data from Akka actors.

[Konrad Malawski (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484753#comment-15484753
 ] 

Konrad Malawski commented on IGNITE-532:


Hi guys,
While I'm not completely up-to-date what this module is aimed to achieve, I 
noticed it seems like integrating with Kafka and streaming, so presumably using 
Akka Streams.
So I just wanted to make sure you're aware of our official Kafka integration: 
https://github.com/akka/reactive-kafka - feel free to use it if it fits your 
needs, it just has hit a stable version last week.

Hope this helps.

-- Konrad (Akka team)

> Implement IgniteAkkaStreamer to stream data from Akka actors.
> -
>
> Key: IGNITE-532
> URL: https://issues.apache.org/jira/browse/IGNITE-532
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Akka|http://akka.io/] for more information. Given that Akka is a Scala 
> frameworks, this streamer should be available in Scala.
> We should create {{IgniteAkkaStreamer}} which will consume messages from Akka 
> Actors and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> * Convert data from Akka to Ignite using an optional pluggable converter. If 
> not provided, then we should have some default mechanism.
> * Specify the cache name for the Ignite cache to load data into.
> * Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-532) Implement IgniteAkkaStreamer to stream data from Akka actors.

[Semen Boikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484647#comment-15484647
 ] 

Semen Boikov commented on IGNITE-532:
-

I'm not sure it makes sense to fix something in current implementation. It 
would be better if you first explain what are you trying to implement and how 
end users will use it.

Thanks

> Implement IgniteAkkaStreamer to stream data from Akka actors.
> -
>
> Key: IGNITE-532
> URL: https://issues.apache.org/jira/browse/IGNITE-532
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Akka|http://akka.io/] for more information. Given that Akka is a Scala 
> frameworks, this streamer should be available in Scala.
> We should create {{IgniteAkkaStreamer}} which will consume messages from Akka 
> Actors and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> * Convert data from Akka to Ignite using an optional pluggable converter. If 
> not provided, then we should have some default mechanism.
> * Specify the cache name for the Ignite cache to load data into.
> * Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-532) Implement IgniteAkkaStreamer to stream data from Akka actors.

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484146#comment-15484146
 ] 

Chandresh Pancholi commented on IGNITE-532:
---

[~sboikov] I will make necessary change and update the PR.


> Implement IgniteAkkaStreamer to stream data from Akka actors.
> -
>
> Key: IGNITE-532
> URL: https://issues.apache.org/jira/browse/IGNITE-532
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Akka|http://akka.io/] for more information. Given that Akka is a Scala 
> frameworks, this streamer should be available in Scala.
> We should create {{IgniteAkkaStreamer}} which will consume messages from Akka 
> Actors and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> * Convert data from Akka to Ignite using an optional pluggable converter. If 
> not provided, then we should have some default mechanism.
> * Specify the cache name for the Ignite cache to load data into.
> * Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-532) Implement IgniteAkkaStreamer to stream data from Akka actors.

[Roman Shtykh (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15469242#comment-15469242
 ] 

Roman Shtykh commented on IGNITE-532:
-

Hi [~Chandresh Pancholi]

_(It seems Semen has started the review, but since I had a look at your code, I 
will write my comments too)_
Putting aside minor issues, in your implementation what the actor is doing is 
just creating a new streamer with data on invocation. To process it, you use 
`AkkaStreamer` that no relation to the created actor. I would recommend to 
revise the code keeping in mind how it will be used the the user.
As I see it, each actor invocation has to inject data into Ignite.


> Implement IgniteAkkaStreamer to stream data from Akka actors.
> -
>
> Key: IGNITE-532
> URL: https://issues.apache.org/jira/browse/IGNITE-532
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Semen Boikov
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Akka|http://akka.io/] for more information. Given that Akka is a Scala 
> frameworks, this streamer should be available in Scala.
> We should create {{IgniteAkkaStreamer}} which will consume messages from Akka 
> Actors and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> * Convert data from Akka to Ignite using an optional pluggable converter. If 
> not provided, then we should have some default mechanism.
> * Specify the cache name for the Ignite cache to load data into.
> * Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-532) Implement IgniteAkkaStreamer to stream data from Akka actors.

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466849#comment-15466849
 ] 

Chandresh Pancholi commented on IGNITE-532:
---

Thank you Semen.

> Implement IgniteAkkaStreamer to stream data from Akka actors.
> -
>
> Key: IGNITE-532
> URL: https://issues.apache.org/jira/browse/IGNITE-532
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Semen Boikov
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Akka|http://akka.io/] for more information. Given that Akka is a Scala 
> frameworks, this streamer should be available in Scala.
> We should create {{IgniteAkkaStreamer}} which will consume messages from Akka 
> Actors and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> * Convert data from Akka to Ignite using an optional pluggable converter. If 
> not provided, then we should have some default mechanism.
> * Specify the cache name for the Ignite cache to load data into.
> * Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-532) Implement IgniteAkkaStreamer to stream data from Akka actors.

[Semen Boikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466817#comment-15466817
 ] 

Semen Boikov commented on IGNITE-532:
-

I start to review PR, will provide comments in next few days.

> Implement IgniteAkkaStreamer to stream data from Akka actors.
> -
>
> Key: IGNITE-532
> URL: https://issues.apache.org/jira/browse/IGNITE-532
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Semen Boikov
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Akka|http://akka.io/] for more information. Given that Akka is a Scala 
> frameworks, this streamer should be available in Scala.
> We should create {{IgniteAkkaStreamer}} which will consume messages from Akka 
> Actors and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> * Convert data from Akka to Ignite using an optional pluggable converter. If 
> not provided, then we should have some default mechanism.
> * Specify the cache name for the Ignite cache to load data into.
> * Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-532) Implement IgniteAkkaStreamer to stream data from Akka actors.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455172#comment-15455172
 ] 

ASF GitHub Bot commented on IGNITE-532:
---

GitHub user chandresh-pancholi opened a pull request:

https://github.com/apache/ignite/pull/1019

IGNITE-532 - Implement IgniteAkkaStreamer to stream data from Akka actors



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chandresh-pancholi/ignite master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1019.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1019


commit 936a105e75c894914867e47cb97df5af62c9316e
Author: chandresh-pancholi 
Date:   2016-09-01T11:51:33Z

IGNITE-532 - Implement IgniteAkkaStreamer to stream data from Akka actors




> Implement IgniteAkkaStreamer to stream data from Akka actors.
> -
>
> Key: IGNITE-532
> URL: https://issues.apache.org/jira/browse/IGNITE-532
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Akka|http://akka.io/] for more information. Given that Akka is a Scala 
> frameworks, this streamer should be available in Scala.
> We should create {{IgniteAkkaStreamer}} which will consume messages from Akka 
> Actors and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> * Convert data from Akka to Ignite using an optional pluggable converter. If 
> not provided, then we should have some default mechanism.
> * Specify the cache name for the Ignite cache to load data into.
> * Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-532) Implement IgniteAkkaStreamer to stream data from Akka actors.

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298716#comment-15298716
 ] 

Chandresh Pancholi commented on IGNITE-532:
---

I have started working on it. I will update once it done.

> Implement IgniteAkkaStreamer to stream data from Akka actors.
> -
>
> Key: IGNITE-532
> URL: https://issues.apache.org/jira/browse/IGNITE-532
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Joshua Goldie
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Akka|http://akka.io/] for more information. Given that Akka is a Scala 
> frameworks, this streamer should be available in Scala.
> We should create {{IgniteAkkaStreamer}} which will consume messages from Akka 
> Actors and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> * Convert data from Akka to Ignite using an optional pluggable converter. If 
> not provided, then we should have some default mechanism.
> * Specify the cache name for the Ignite cache to load data into.
> * Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-532) Implement IgniteAkkaStreamer to stream data from Akka actors.

[Mahoud (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15218344#comment-15218344
 ] 

Mahoud commented on IGNITE-532:
---

Hi All, any updates for that topic , as we are working into a new project which 
is using akka for our order manager and we will validate the possibility of 
using Ignite as the data and service grid behind , and i am wondering is it 
possible to replace akka cluster manager with ignite one or at least if there 
is data grid integration via streams , i can can contribute the needed work as 
well if you want.

Thanks a lot,
Romeh

> Implement IgniteAkkaStreamer to stream data from Akka actors.
> -
>
> Key: IGNITE-532
> URL: https://issues.apache.org/jira/browse/IGNITE-532
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Joshua Goldie
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Akka|http://akka.io/] for more information. Given that Akka is a Scala 
> frameworks, this streamer should be available in Scala.
> We should create {{IgniteAkkaStreamer}} which will consume messages from Akka 
> Actors and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> * Convert data from Akka to Ignite using an optional pluggable converter. If 
> not provided, then we should have some default mechanism.
> * Specify the cache name for the Ignite cache to load data into.
> * Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)