[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Roman Shtykh (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15104121#comment-15104121
 ] 

Roman Shtykh commented on IGNITE-429:
-

Anton,

The suggestion is submitted.


> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Roman Shtykh
> Fix For: 1.6
>
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101666#comment-15101666
 ] 

Anton Vinogradov commented on IGNITE-429:
-

Roman, 

I've updated 'Streamers' & created empty StormStreamer page at readme.io 1.6 
(https://apacheignite.readme.io/v1.6/docs/apache-storm-streamer) please suggest 
edit.
Also I've merged changes to master.

Thanks for contribution!

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Roman Shtykh
> Fix For: 1.6
>
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Roman Shtykh (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101796#comment-15101796
 ] 

Roman Shtykh commented on IGNITE-429:
-

Anton,

Thanks a lot!
I will probably be able to suggest the contents for the page over the week-end.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Roman Shtykh
> Fix For: 1.6
>
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15097833#comment-15097833
 ] 

Anton Vinogradov commented on IGNITE-429:
-

Roman, 
That's a regular check for streamers, we should ensure that only corect data 
goes to the cache, easiest way is to check before and after :) 
First check should happened after components created for test but before they 
should start data producing.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Roman Shtykh
> Fix For: 1.6
>
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15097944#comment-15097944
 ] 

Anton Vinogradov commented on IGNITE-429:
-

Roman,

Looks good, please check "Checklist before push" ( 
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-Checklistbeforepush
 ) and I'll push changes to master.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Roman Shtykh
> Fix For: 1.6
>
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Roman Shtykh (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15097867#comment-15097867
 ] 

Roman Shtykh commented on IGNITE-429:
-

Anton, thanks, I have updated PR. I added a check for cache is empty after the 
streamer is created && before the topology is started.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Roman Shtykh
> Fix For: 1.6
>
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Roman Shtykh (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15100385#comment-15100385
 ] 

Roman Shtykh commented on IGNITE-429:
-

Anton,

Run through the check list, warnings fixed. Unfortunately, I don't have 
permissions to add it to 'Streamers' on TC. May I ask you to add it?

As to 'Documentation & examples,' I will write an article on how to use it 
based on README.txt. Should I do it on readme.io (under ver. 1.5 I guess) or 
prepare a wiki file and attach it here?

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Roman Shtykh
> Fix For: 1.6
>
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Roman Shtykh (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15097647#comment-15097647
 ] 

Roman Shtykh commented on IGNITE-429:
-

Anton, thank you! Very good point on IGNITE_TUPLE_FIELD.

In 2, do you mean I have to check the cache is empty before the test data is 
streamed via Storm? Why?

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Roman Shtykh
> Fix For: 1.6
>
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15096340#comment-15096340
 ] 

Anton Vinogradov commented on IGNITE-429:
-

Roman I've made minor fixes:
https://github.com/avinogradovgg/ignite/commit/3beab7debdfaa7ac6962ed4bdd33e40fea5f067e

Also, 
1) Seems that IGNITE_TUPLE_FIELD should be configurable.
2) Cache content should be cheched before and after test.


> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Roman Shtykh
> Fix For: 1.6
>
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093669#comment-15093669
 ] 

Anton Vinogradov commented on IGNITE-429:
-

Roman, could you please update pull request to fix merge conflicts with master?

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Roman Shtykh
> Fix For: 1.6
>
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Roman Shtykh (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15095407#comment-15095407
 ] 

Roman Shtykh commented on IGNITE-429:
-

The conflicts are resolved now.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Roman Shtykh
> Fix For: 1.6
>
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Denis Magda (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15089229#comment-15089229
 ] 

Denis Magda commented on IGNITE-429:


Roman, thanks! Someone will review the changes the next week.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Roman Shtykh
> Fix For: 1.6
>
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Roman Shtykh (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15083076#comment-15083076
 ] 

Roman Shtykh commented on IGNITE-429:
-

Fixed the above-mention limitation. Storm parallelization is usable now.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Roman Shtykh
> Fix For: 1.6
>
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082519#comment-15082519
 ] 

ASF GitHub Bot commented on IGNITE-429:
---

GitHub user shroman opened a pull request:

https://github.com/apache/ignite/pull/393

IGNITE-429: Apache Storm Streamer



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shroman/ignite ignite-429

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/393.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #393


commit 8420e109c4d07bbf438f17dc79268c39b47e76b2
Author: chandresh pancholi 
Date:   2015-09-15T12:50:30Z

ignite-429 feature fix

commit a5adeb4c7badd52dd7724fa5a9a9ba7924e934ed
Author: chandresh pancholi 
Date:   2015-09-16T13:19:37Z

Ignite code refactor and modification

commit c6335046a07d9859a46d078e6cd8d7d2a7011df2
Author: chandresh pancholi 
Date:   2015-09-21T11:59:24Z

Code refactor

commit 4596bedfbfc756d3974b4d2270401515fa618bb5
Author: chandresh pancholi 
Date:   2015-09-28T09:12:13Z

Work completed

commit 538986b186b3d89797a34fe35c8120fecaf9
Author: chandresh pancholi 
Date:   2015-09-29T11:37:17Z

Apache ignite-storm integration

commit 3f40de788e5a050bbefb69ac2517a625e72b5299
Author: chandresh pancholi 
Date:   2015-09-29T11:37:50Z

exclusion of zookeeper

commit aa59cdade93d41d877b1966c4851aa827981b73c
Author: chandresh pancholi 
Date:   2015-10-03T12:13:18Z

Implemented Refactor suggestion given by Anton in Ignite0429

commit 2e103b915ba88b38604030092f55ed1f04b14cfe
Author: chandresh pancholi 
Date:   2015-10-05T07:43:02Z

Code refactor in storm module suggested by Gian/Anton

commit 7fa633cfb0cfb8144e780d9572a3a4fbe518bbd4
Author: chandresh pancholi 
Date:   2015-10-07T08:52:12Z

Adding javadoc in StormSpout class and make streamer compatible with java1.7

commit c181e3d7bf55f91cc0ccfa0407fae72f4c1274c3
Author: chandresh pancholi 
Date:   2015-10-07T13:18:44Z

added assertEqual in Storm tests

commit 96b1095f3b6bfbaedb02974c66acbca2cfc8c9bf
Author: chandresh pancholi 
Date:   2015-10-07T13:19:52Z

added assertEqual in Storm tests

commit d11a0d89aab610998cfc32fca8ddb9b2e01d7854
Author: chandresh pancholi 
Date:   2015-10-09T08:07:10Z

Remove no. of  workers from test class and changed the time out

commit 67baeb940006e5054dd7b6c4994a2907071ecb68
Author: chandresh pancholi 
Date:   2015-10-19T11:53:24Z

Generation of another node in ignite unit test

commit 39ef112dacbd410d6955641fa59d74b9c691e91f
Author: Anton Vinogradov 
Date:   2015-11-23T10:58:05Z

Merge remote-tracking branch 'remotes/origin/ignite-1.5' into 
ignite-429-pull-fixes

Conflicts:
pom.xml

commit 39f5e40952c5e636b1116a200921df2258a6e27a
Author: Anton Vinogradov 
Date:   2015-11-23T10:58:45Z

Merge remote-tracking branch 'remotes/origin/ignite-1.5' into 
ignite-429-pull-fixes

Conflicts:
pom.xml

commit d67994e381e83cd6bda627925e561c15ad2fe5c3
Author: Anton Vinogradov 
Date:   2015-11-23T10:59:17Z

429 Pull request fixes

commit 4127c22a99136bd04850ee35c7181d4c49193393
Author: shtykh_roman 
Date:   2016-01-05T05:54:49Z

IGNITE-429: IgniteStormStreamer implementation reworked.

commit bae4d447ad694bcd1daa9795ba8223d4d57029ca
Author: shtykh_roman 
Date:   2016-01-05T06:42:50Z

IGNITE-429: IgniteStormStreamer implementation reworked. Changes test 
configuration.




> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Roman Shtykh
> Fix For: 1.6
>
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create 

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15021959#comment-15021959
 ] 

Anton Vinogradov commented on IGNITE-429:
-

Chandresh, 

I've made some fixes to make your test runnable.
see changes at 
https://github.com/avinogradovgg/ignite/commit/d67994e381e83cd6bda627925e561c15ad2fe5c3
 .
also I've added 
https://github.com/avinogradovgg/ignite/blob/ignite-429-pull-fixes/modules/storm/src/test/resources/example-ignite.xml
 at previous commit.

Now test hangs on cluster.shutdown(); Please investigate reasons.

Also, Please carefully review other streamers before continue work on this 
issue. Twitter streamer very similar to Storm streamer.
Feel free to copy code & design from another streamers. 

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14998224#comment-14998224
 ] 

Anton Vinogradov commented on IGNITE-429:
-

Thanks for info.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14998222#comment-14998222
 ] 

Anton Vinogradov commented on IGNITE-429:
-

Thanks for info.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14996525#comment-14996525
 ] 

Anton Vinogradov commented on IGNITE-429:
-

Chandresh, 
Thanks, I've started investigation/refactoring. 
Checked that test hang on each run. Seems I need to read more about Storm 
before changes ;) 

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Roman Shtykh (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997911#comment-14997911
 ] 

Roman Shtykh commented on IGNITE-429:
-

Anton,
(To save your time)

Storm testing framework times out after 5 secs. To avoid it, you need set 
STORM_TEST_TIMEOUT_MS env. variable.
{code}
$ export STORM_TEST_TIMEOUT_MS=1
$ mvn test
{code}


> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997289#comment-14997289
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

Before that can you check whats the mistake am i making with ignite node that i 
am starting?
One node i am starting in beforetest and another one is in StormStreamer class.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14995609#comment-14995609
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

Anton,
Here is the request https://github.com/apache/ignite/pull/138



> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991455#comment-14991455
 ] 

Anton Vinogradov commented on IGNITE-429:
-

Chandresh,
I'm going to help you with this task. Could you please update your pull request 
with your latest changes? 
I'll fix Coding Guidelines issues and try to help with tests.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991467#comment-14991467
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

Thanks Anton.
I have been facing issue with another node creation.
Will send pull request .

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14962908#comment-14962908
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

Everytime i run tests i get Test timed out (5000ms) (not (every? (memfn 
waiting?) daemons)).

I tried with possible timeouts form 5 sec to 50 sec. :( 

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14962957#comment-14962957
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

Gian,
Where to put this change  Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14963189#comment-14963189
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

Gian,
I made changes in config/default-config.xml . i am still getting same 
exception. can you please check where am i making mistake?
I have pushed the code in my 
branch(https://github.com/chandresh-pancholi/ignite)

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Gianfranco Murador (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14963308#comment-14963308
 ] 

Gianfranco Murador commented on IGNITE-429:
---

Chan , could you post the stack trace of the error ? Also are you sure you're 
using the same config to start two different instance of ignite ?

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14963955#comment-14963955
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

Hey Gian, i am using same config to start another node.
Here is the stack trace
org.apache.ignite.IgniteException: Default grid instance has already been 
started.
at 
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:880)
 ~[classes/:na]
at org.apache.ignite.Ignition.start(Ignition.java:349) ~[classes/:na]
at 
org.apache.ignite.stream.storm.StormStreamer.prepare(StormStreamer.java:155) 
~[classes/:na]
at 
backtype.storm.daemon.executor$fn__3439$fn__3451.invoke(executor.clj:692) 
~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.util$async_loop$fn__460.invoke(util.clj:461) 
~[storm-core-0.9.5.jar:0.9.5]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
Caused by: org.apache.ignite.IgniteCheckedException: Default grid instance has 
already been started.
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:952) 
~[classes/:na]
at 
org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:892) 
~[classes/:na]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:784) 
~[classes/:na]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:705) 
~[classes/:na]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:576) 
~[classes/:na]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:546) 
~[classes/:na]
at org.apache.ignite.Ignition.start(Ignition.java:346) ~[classes/:na]
... 5 common frames omitted
29332 [Thread-11-bolt] ERROR backtype.storm.daemon.executor - 
org.apache.ignite.IgniteException: Default grid instance has already been 
started.
at 
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:880)
 ~[classes/:na]
at org.apache.ignite.Ignition.start(Ignition.java:349) ~[classes/:na]
at 
org.apache.ignite.stream.storm.StormStreamer.prepare(StormStreamer.java:155) 
~[classes/:na]
at 
backtype.storm.daemon.executor$fn__3439$fn__3451.invoke(executor.clj:692) 
~[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.util$async_loop$fn__460.invoke(util.clj:461) 
~[storm-core-0.9.5.jar:0.9.5]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
Caused by: org.apache.ignite.IgniteCheckedException: Default grid instance has 
already been started.
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:952) 
~[classes/:na]
at 
org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:892) 
~[classes/:na]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:784) 
~[classes/:na]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:705) 
~[classes/:na]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:576) 
~[classes/:na]
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:546) 
~[classes/:na]
at org.apache.ignite.Ignition.start(Ignition.java:346) ~[classes/:na]
... 5 common frames omitted
29351 [Thread-11-bolt] ERROR backtype.storm.util - Halting process: ("Worker 
died")
java.lang.RuntimeException: ("Worker died")
at backtype.storm.util$exit_process_BANG_.doInvoke(util.clj:325) 
[storm-core-0.9.5.jar:0.9.5]
at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.5.1.jar:na]
at 
backtype.storm.daemon.worker$fn__4694$fn__4695.invoke(worker.clj:493) 
[storm-core-0.9.5.jar:0.9.5]
at 
backtype.storm.daemon.executor$mk_executor_data$fn__3272$fn__3273.invoke(executor.clj:240)
 [storm-core-0.9.5.jar:0.9.5]
at backtype.storm.util$async_loop$fn__460.invoke(util.clj:473) 
[storm-core-0.9.5.jar:0.9.5]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
[23:51:33,482][INFO ][Thread-2][G] Invoking shutdown hook...
[23:51:33,483][INFO ][Thread-1][G] Invoking shutdown hook...
[23:51:33,495][INFO ][Thread-2][GridTcpRestProtocol] Command protocol 
successfully stopped: TCP binary
[23:51:33,578][INFO ][Thread-2][GridCacheProcessor] Stopped cache: 
ignite-marshaller-sys-cache
[23:51:33,578][INFO ][Thread-2][GridCacheProcessor] Stopped cache: 
ignite-sys-cache
[23:51:33,578][INFO ][Thread-2][GridCacheProcessor] Stopped cache: 
ignite-atomics-sys-cache
[23:51:33,579][INFO ][Thread-2][GridCacheProcessor] Stopped cache: igniteCache


> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: 

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Gianfranco Murador (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14962481#comment-14962481
 ] 

Gianfranco Murador commented on IGNITE-429:
---

Hello Chandresh,
 to solve that problem simply assign a different cache in the configuration 
file : 
 
However I believe that it is almost impossible to take a test within 5 seconds.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14950050#comment-14950050
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

How to check data added in streamer.

StormStreamer.java
getStreamer().addData(k, igniteGrid.get(k));

When i try fetching data in Test class i am getting Null.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Gianfranco Murador (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14950055#comment-14950055
 ] 

Gianfranco Murador commented on IGNITE-429:
---

Hi Chandresh,  
I think that the test is more complex, in my opinion, 
You should start another node , insert data into the grid with the streamer and 
then retrieve the same grid and ensure that the same data were included. From 
the moment the single node is stopped we also lose the data entered in the grid.
I think it's difficult to get the test within 5 sec.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14950036#comment-14950036
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

Ignore above stack trace.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946505#comment-14946505
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

[~glutters] Tests are continuously timing out  for me. Can you please check 
again?



> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946541#comment-14946541
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

GitHub user chandresh-pancholi opened a pull request:

https://github.com/apache/ignite/pull/138

 [Ignite-429] Implement IgniteStormStreamer to stream data from Apache Storm

Added java1.7 compatibility and javadoc in StormSpout class.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chandresh-pancholi/ignite master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/138.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #138
commit 7fa633cfb0cfb8144e780d9572a3a4fbe518bbd4
Author: chandresh pancholi 
Date:   2015-10-07T08:52:12Z

Adding javadoc in StormSpout class and make streamer compatible with java1.7



> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946561#comment-14946561
 ] 

Anton Vinogradov commented on IGNITE-429:
-

Chandresh,

Next time please update existing pull request instead of creation new one.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946606#comment-14946606
 ] 

Anton Vinogradov commented on IGNITE-429:
-

Chandresh,

is https://github.com/apache/ignite/pull/138 ready to be reviewed?

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Roman Shtykh (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946948#comment-14946948
 ] 

Roman Shtykh commented on IGNITE-429:
-

Chandresh,

I cannot run your code right now and not sure where you get a timeout, but if 
it is related to TEST-TIMEOUT-MS of the topology, see
CompleteTopologyParam's setTimeoutMs()

I think it is implemented in Storm 0.10.0. In earlier versions this test 
parameter is not accessible from Java.

So changing Storm version and checking completeTopologyParam.setTimeoutMs(...) 
may help.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Roman Shtykh (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946950#comment-14946950
 ] 

Roman Shtykh commented on IGNITE-429:
-

Default timeout is 5000 ms.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945004#comment-14945004
 ] 

Anton Vinogradov commented on IGNITE-429:
-

Chandresh, following problems still actual at pr-130:
1) Code contains redundant comments.
for example: // startTopology(stormStreamer);
2) Redundant newlines should be removed. At code and imports.
3) Newlines should be added where it necessary:
for example:
if (log.isDebugEnabled()) {
log.info("get tuple from storm  " + k + ", " + igniteGrid.get(k));
}
getStreamer().addData(k, igniteGrid.get(k));
5) @Override should be located according to Coding Guidelines.
6) Javadoc should present for each method and sentences should be finished with 
dot.
7) Code should be java7 compatible.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945006#comment-14945006
 ] 

Anton Vinogradov commented on IGNITE-429:
-

I've updated 
https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines 
Hope it will help.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944983#comment-14944983
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

Anton,

Didn't get you. What do you mean by actual code?

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943993#comment-14943993
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

GitHub user chandresh-pancholi opened a pull request:

https://github.com/apache/ignite/pull/130

[Ignite-429] Implement IgniteStormStreamer to stream data from Apache Storm

I have made some refactor changes suggested by Gian/Anton.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chandresh-pancholi/ignite master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/130.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #130


commit 8420e109c4d07bbf438f17dc79268c39b47e76b2
Author: chandresh pancholi 
Date:   2015-09-15T12:50:30Z

ignite-429 feature fix

commit a5adeb4c7badd52dd7724fa5a9a9ba7924e934ed
Author: chandresh pancholi 
Date:   2015-09-16T13:19:37Z

Ignite code refactor and modification

commit c6335046a07d9859a46d078e6cd8d7d2a7011df2
Author: chandresh pancholi 
Date:   2015-09-21T11:59:24Z

Code refactor

commit 4596bedfbfc756d3974b4d2270401515fa618bb5
Author: chandresh pancholi 
Date:   2015-09-28T09:12:13Z

Work completed

commit 538986b186b3d89797a34fe35c8120fecaf9
Author: chandresh pancholi 
Date:   2015-09-29T11:37:17Z

Apache ignite-storm integration

commit 3f40de788e5a050bbefb69ac2517a625e72b5299
Author: chandresh pancholi 
Date:   2015-09-29T11:37:50Z

exclusion of zookeeper

commit aa59cdade93d41d877b1966c4851aa827981b73c
Author: chandresh pancholi 
Date:   2015-10-03T12:13:18Z

Implemented Refactor suggestion given by Anton in Ignite0429

commit 2e103b915ba88b38604030092f55ed1f04b14cfe
Author: chandresh pancholi 
Date:   2015-10-05T07:43:02Z

Code refactor in storm module suggested by Gian/Anton



> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943314#comment-14943314
 ] 

Anton Vinogradov commented on IGNITE-429:
-

Chandresh,

Please don't forget to mention new pull request's url & change issue status to 
"Patch Available" when it will be ready for review.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Gianfranco Murador (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942617#comment-14942617
 ] 

Gianfranco Murador commented on IGNITE-429:
---

Hello Chandresh, 
the code has been written correctly, 
can you please check these tips described by Anton ? 
Regards, Gianfranco

2) Redundant newlines should be removed. At code and imports.
3) Newlines should be added where it necessary:
for example:
Map res = process(tuple);
collector.emit(new Values(res));
5) @Override should be located according to Coding Guidelines.
6) Javadoc sentences should be finished with dot.


> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Gianfranco Murador (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942618#comment-14942618
 ] 

Gianfranco Murador commented on IGNITE-429:
---

Hello Chandresh, 
the code has been written correctly, 
can you please check these tips described by Anton ? 
Regards, Gianfranco

2) Redundant newlines should be removed. At code and imports.
3) Newlines should be added where it necessary:
for example:
Map res = process(tuple);
collector.emit(new Values(res));
5) @Override should be located according to Coding Guidelines.
6) Javadoc sentences should be finished with dot.


> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942638#comment-14942638
 ] 

ASF GitHub Bot commented on IGNITE-429:
---

Github user murador closed the pull request at:

https://github.com/apache/ignite/pull/110


> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942267#comment-14942267
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

Gian can you review my code before i make pull request?

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939637#comment-14939637
 ] 

Anton Vinogradov commented on IGNITE-429:
-

Gianfranco,

Seems that issue should change Assignee. Could you please assign issue to 
yourself?

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939682#comment-14939682
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

Sure Gian!

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Gianfranco Murador (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939679#comment-14939679
 ] 

Gianfranco Murador commented on IGNITE-429:
---

Chan, 
 As the ticket is assigned to you, you can make any changes in your own branch 
on your fork and make a pull request , you can start with all the changes that 
you see in my fork. 

Anton,  thanks for the tips. 
Regards, Gianfranco 



> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Anton Vinogradov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939618#comment-14939618
 ] 

Anton Vinogradov commented on IGNITE-429:
-

Gianfranco,

Thanks for fixing pull request,

Sorry for delay, but I did not see notifications that you made changes. You 
sent notification to @anton-vinogradov, but that's not my primary github 
account.
Next time please update issue, in this case every community member will be 
informed.

Unfortunately code still not ready to be reviewed.
1) Code contains redundant comments.
for example: // refreshPartitions();
2) Redindant newlines should be removed. At code and imports.
3) Newlines should be added where it necessary:
for example:
Map res = process(tuple);
collector.emit(new Values(res));
4) Some methods have no javadoc.
5) @Override should be located according to Coding Guidelines.
6) Javadoc sentences should be finished with dot.
7) Git conflicts should be resolved, see GridSqlQueryParser.java for details.

Please recheck whole code matches Coding Guidelines and contains only files 
related to this issue.

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14910199#comment-14910199
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

I will update wiki document .

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14933284#comment-14933284
 ] 

ASF GitHub Bot commented on IGNITE-429:
---

Github user murador closed the pull request at:

https://github.com/apache/ignite/pull/107


> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14933311#comment-14933311
 ] 

ASF GitHub Bot commented on IGNITE-429:
---

GitHub user murador opened a pull request:

https://github.com/apache/ignite/pull/110

IGNITE-429:  Implement IgniteStormStreamer to stream data from Apache Storm

The test was successful with the latest changes, we use a simulated 
topology. The operation of the stream is different compared to the one 
described in the guidelines because the integration is via a Storm Bolt. The 
operation will be described here : 
https://cwiki.apache.org/confluence/display/IGNITE/Storm+Integration+Guidelines

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/murador/ignite IGNITE-429

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/110.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #110


commit a7a7e9b1703c4a8a3bcd805893f88c05af1903ae
Author: murador 
Date:   2015-09-13T16:51:19Z

IGNITE-708 remove refreshPartitions on ResendTimeoutObject

commit bf3070b838fbdedbb95b8e0bf95ffbef48aeb71b
Author: glutters 
Date:   2015-09-19T14:22:19Z

IGNITE-429 definition of a draft of  StormStreamer class

commit 29b889abff0056b5906cdee9843d8e177fb8da40
Author: glutters 
Date:   2015-09-19T14:22:43Z

IGNITE-429 definition of a draft of  StormStreamer class 2

commit f15be0799c80429319ec1a4164f3fb89ff57ec34
Author: glutters 
Date:   2015-09-19T15:03:03Z

IGNITE-429 definition of a draft of StormStreamer with mod

commit f8c46d0d6638856850aab56903f5b0555c98785c
Author: glutters 
Date:   2015-09-19T16:06:18Z

IGNITE-429 Ignite Bolt included

commit 886abb919b92dd399f96370a2f3a22467187a98a
Author: glutters 
Date:   2015-09-19T16:32:08Z

Sinc of the repo

commit 43f1f9c3063e30f950e593ac36e438152b60a3c1
Author: glutters 
Date:   2015-09-20T10:11:03Z

IGNITE-429 - first commit on branch

commit ba59921b99beee5e527a56822cf34dad669e75de
Author: glutters 
Date:   2015-09-20T13:41:51Z

IGNITE-429 - finish implementation - going to unit test

commit 11eafbc09ece64d740639b752da0e39471833919
Author: Gianfranco Murador 
Date:   2015-09-20T13:45:29Z

Ugly Linux backup file

Ugly Linux backup file

commit e9ff1274abdd9feb0a50713e0a6ba9c4380f205a
Author: glutters 
Date:   2015-09-20T14:13:24Z

IGNITE-429 - fix

commit 693882e9bb34adced8a98c6cf84c8fd11836b8ab
Author: glutters 
Date:   2015-09-20T14:16:11Z

Merge branch 'IGNITE-429' of https://github.com/murador/ignite into 
IGNITE-429

commit 3e02c2a7918db0b6756b0fff0058fc4e22f05abb
Author: glutters 
Date:   2015-09-26T13:43:39Z

IGNITE-429 - unit test created

commit 9ace43739306b68eba6bf7e0cc818ecadbb5befe
Author: glutters 
Date:   2015-09-26T14:46:16Z

fix compilatio

commit 71994a71f58cf616ecc3d8d55e4d1ea5f29fcaef
Author: glutters 
Date:   2015-09-27T10:57:27Z

IGNITE-429 completed testing

commit 0f29871b54d6bf621db1b2efe95f32c6531c8527
Author: glutters 
Date:   2015-09-27T10:57:56Z

IGNITE-429 completed

commit 28dfa98ca204a7cfd2f1143e7b55afaebc871b7a
Author: glutters 
Date:   2015-09-27T11:04:24Z

IGNITE-429 completed

commit 87db3121d1f37ccaa395f6af90c9eb56c1567c51
Author: glutters 
Date:   2015-09-27T12:09:57Z

IGNITE-429 completed

commit 7d359e272d83d861bd4cab995778528388faa8db
Author: glutters 
Date:   2015-09-27T12:31:25Z

IGNITE-429 completed

commit a4578dab26489db673a2e1eb573b59a509a88e85
Author: glutters 
Date:   2015-09-27T12:38:34Z

IGNITE-429 completed

commit a5ea969f1f2cb8d0475f8da720c97bb94eb008e7
Author: glutters 
Date:   2015-09-27T12:46:23Z

IGNITE-429 completed

commit 46230b6afff7bf6f87d0af6b37c3dc44af8e91c7
Author: glutters 
Date:   2015-09-27T14:37:22Z

merge commit

commit 8cf75a9f7937c9a28c846c916dfcdaa4ee2e9298
Author: glutters 
Date:   2015-09-28T10:42:29Z

Fix timeout in unit test

commit 9f48ce40efa2949ec3d5a5aadf73d473d325e7bf
Author: glutters 
Date:   2015-09-28T13:21:31Z

IGNITE-429 - set up unit test with simulated topology




> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: 

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14909724#comment-14909724
 ] 

ASF GitHub Bot commented on IGNITE-429:
---

GitHub user murador opened a pull request:

https://github.com/apache/ignite/pull/107

IGNITE-429:  Implement IgniteStormStreamer to stream data from Apache Storm

The test was successful with the latest changes . The operation of the 
stream is different compared to the one described in the guidelines because the 
integration is via a Storm Bolt. The operation will be described here : 
https://cwiki.apache.org/confluence/display/IGNITE/Storm+Integration+Guidelines

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/murador/ignite IGNITE-429

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/107.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #107


commit a7a7e9b1703c4a8a3bcd805893f88c05af1903ae
Author: murador 
Date:   2015-09-13T16:51:19Z

IGNITE-708 remove refreshPartitions on ResendTimeoutObject

commit bf3070b838fbdedbb95b8e0bf95ffbef48aeb71b
Author: glutters 
Date:   2015-09-19T14:22:19Z

IGNITE-429 definition of a draft of  StormStreamer class

commit 29b889abff0056b5906cdee9843d8e177fb8da40
Author: glutters 
Date:   2015-09-19T14:22:43Z

IGNITE-429 definition of a draft of  StormStreamer class 2

commit f15be0799c80429319ec1a4164f3fb89ff57ec34
Author: glutters 
Date:   2015-09-19T15:03:03Z

IGNITE-429 definition of a draft of StormStreamer with mod

commit f8c46d0d6638856850aab56903f5b0555c98785c
Author: glutters 
Date:   2015-09-19T16:06:18Z

IGNITE-429 Ignite Bolt included

commit 886abb919b92dd399f96370a2f3a22467187a98a
Author: glutters 
Date:   2015-09-19T16:32:08Z

Sinc of the repo

commit 43f1f9c3063e30f950e593ac36e438152b60a3c1
Author: glutters 
Date:   2015-09-20T10:11:03Z

IGNITE-429 - first commit on branch

commit ba59921b99beee5e527a56822cf34dad669e75de
Author: glutters 
Date:   2015-09-20T13:41:51Z

IGNITE-429 - finish implementation - going to unit test

commit 11eafbc09ece64d740639b752da0e39471833919
Author: Gianfranco Murador 
Date:   2015-09-20T13:45:29Z

Ugly Linux backup file

Ugly Linux backup file

commit e9ff1274abdd9feb0a50713e0a6ba9c4380f205a
Author: glutters 
Date:   2015-09-20T14:13:24Z

IGNITE-429 - fix

commit 693882e9bb34adced8a98c6cf84c8fd11836b8ab
Author: glutters 
Date:   2015-09-20T14:16:11Z

Merge branch 'IGNITE-429' of https://github.com/murador/ignite into 
IGNITE-429

commit 3e02c2a7918db0b6756b0fff0058fc4e22f05abb
Author: glutters 
Date:   2015-09-26T13:43:39Z

IGNITE-429 - unit test created

commit 9ace43739306b68eba6bf7e0cc818ecadbb5befe
Author: glutters 
Date:   2015-09-26T14:46:16Z

fix compilatio

commit 71994a71f58cf616ecc3d8d55e4d1ea5f29fcaef
Author: glutters 
Date:   2015-09-27T10:57:27Z

IGNITE-429 completed testing

commit 0f29871b54d6bf621db1b2efe95f32c6531c8527
Author: glutters 
Date:   2015-09-27T10:57:56Z

IGNITE-429 completed

commit 28dfa98ca204a7cfd2f1143e7b55afaebc871b7a
Author: glutters 
Date:   2015-09-27T11:04:24Z

IGNITE-429 completed

commit 87db3121d1f37ccaa395f6af90c9eb56c1567c51
Author: glutters 
Date:   2015-09-27T12:09:57Z

IGNITE-429 completed

commit 7d359e272d83d861bd4cab995778528388faa8db
Author: glutters 
Date:   2015-09-27T12:31:25Z

IGNITE-429 completed

commit a4578dab26489db673a2e1eb573b59a509a88e85
Author: glutters 
Date:   2015-09-27T12:38:34Z

IGNITE-429 completed

commit a5ea969f1f2cb8d0475f8da720c97bb94eb008e7
Author: glutters 
Date:   2015-09-27T12:46:23Z

IGNITE-429 completed




> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will 

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Gianfranco Murador (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14909691#comment-14909691
 ] 

Gianfranco Murador commented on IGNITE-429:
---

The test was successful with the latest changes . The operation of the stream 
is different compared to the one described in the guidelines because the 
integration is via a Storm Bolt . I ask Chandresh provide documentation about  
so that we can include in wiki .

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900558#comment-14900558
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

Gian,

I have included  your suggestions and wrote unit test as well but i am having 
problem with getStreamer() and getIgnite(). Its giving me null.

I have pushed the code(https://github.com/chandresh-pancholi/ignite). can you 
please check?

Thanks

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Gianfranco Murador (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877480#comment-14877480
 ] 

Gianfranco Murador commented on IGNITE-429:
---

Hello Chandresh,
how do you see the implementation is simple ( this is good ), but It does not 
take into account of some aspects that may not work in Ignite. As we have done 
in Kafka streamer is a good idea to create an ExecutorService to submit streams 
from apache storm (This improves performance) . You have to rely to a naming 
convention that must be declared with your solution, while it might be a good 
idea to create our bolt to filter results. I will make a new branch with all 
the changes, so you can make a new pull request. Still I am not very skilled 
with git :).
Regards, Gianfranco 

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Gianfranco Murador (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14899944#comment-14899944
 ] 

Gianfranco Murador commented on IGNITE-429:
---

Hi Chandresh, 
  please, look here 
https://github.com/murador/ignite/tree/IGNITE-429/modules/storm/src. Now, We 
need to create unit test.Let me know if you find problems with the proposed 
solution .

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877242#comment-14877242
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

Hi Gian,
If you have some time,can you go through my code and check where am i making 
mistake?

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Chandresh Pancholi (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745372#comment-14745372
 ] 

Chandresh Pancholi commented on IGNITE-429:
---

It would be great if you provide me your implementation. is it already pushed?

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm

[Gianfranco Murador (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745419#comment-14745419
 ] 

Gianfranco Murador commented on IGNITE-429:
---

Hi Chandresh, 
I will commit my solution on my fork by this weekend , so you can verify the 
two solutions.
Thank you for your contribution

> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
>  Issue Type: Sub-task
>  Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under 
> high load. It was previously named {{IgniteDataLoader}}, see ticket 
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm 
> and stream them into Ignite caches. 
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable 
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)