[jira] [Comment Edited] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm
[
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15100385#comment-15100385
]
Roman Shtykh edited comment on IGNITE-429 at 1/15/16 6:42 AM:
--
Anton,
Ran through the check list, warnings fixed. Unfortunately, I don't have
permissions to add it to 'Streamers' on TC. May I ask you to add it?
As to 'Documentation & examples,' I will write an article on how to use it
based on README.txt. Should I do it on readme.io (under ver. 1.5 I guess) or
prepare a wiki file and attach it here?
was (Author: roman_s):
Anton,
Run through the check list, warnings fixed. Unfortunately, I don't have
permissions to add it to 'Streamers' on TC. May I ask you to add it?
As to 'Documentation & examples,' I will write an article on how to use it
based on README.txt. Should I do it on readme.io (under ver. 1.5 I guess) or
prepare a wiki file and attach it here?
> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
> Issue Type: Sub-task
> Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Roman Shtykh
> Fix For: 1.6
>
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under
> high load. It was previously named {{IgniteDataLoader}}, see ticket
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm
> and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm
[
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14962957#comment-14962957
]
Chandresh Pancholi edited comment on IGNITE-429 at 10/19/15 7:25 AM:
-
Gian,
Where to put this change Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
> Issue Type: Sub-task
> Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under
> high load. It was previously named {{IgniteDataLoader}}, see ticket
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm
> and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm
[
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14962957#comment-14962957
]
Chandresh Pancholi edited comment on IGNITE-429 at 10/19/15 9:44 AM:
-
Gian,
Where to put this change Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
> Issue Type: Sub-task
> Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under
> high load. It was previously named {{IgniteDataLoader}}, see ticket
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm
> and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm
[
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939679#comment-14939679
]
Gianfranco Murador edited comment on IGNITE-429 at 10/2/15 9:35 AM:
Chan,
As the ticket is assigned to you, you can make any changes in your own branch
on your fork and make a pull request , you can start with all changes that you
see in my fork.
Anton, thanks for the tips.
Regards, Gianfranco
was (Author: glutters):
Chan,
As the ticket is assigned to you, you can make any changes in your own branch
on your fork and make a pull request , you can start with all the changes that
you see in my fork.
Anton, thanks for the tips.
Regards, Gianfranco
> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
> Issue Type: Sub-task
> Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under
> high load. It was previously named {{IgniteDataLoader}}, see ticket
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm
> and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm
[
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939618#comment-14939618
]
Anton Vinogradov edited comment on IGNITE-429 at 10/1/15 10:18 AM:
---
Gianfranco,
Thanks for fixing pull request,
Sorry for delay, but I did not see notifications that you made changes. You
sent notification to @anton-vinogradov, but that's not my primary github
account.
Next time please update issue, in this case every community member will be
informed.
Unfortunately code still not ready to be reviewed.
1) Code contains redundant comments.
for example: // refreshPartitions();
2) Redundant newlines should be removed. At code and imports.
3) Newlines should be added where it necessary:
for example:
Map res = process(tuple);
collector.emit(new Values(res));
4) Some methods have no javadoc.
5) @Override should be located according to Coding Guidelines.
6) Javadoc sentences should be finished with dot.
7) Git conflicts should be resolved, see GridSqlQueryParser.java for details.
Please recheck whole code matches Coding Guidelines and contains only files
related to this issue.
was (Author: avinogradov):
Gianfranco,
Thanks for fixing pull request,
Sorry for delay, but I did not see notifications that you made changes. You
sent notification to @anton-vinogradov, but that's not my primary github
account.
Next time please update issue, in this case every community member will be
informed.
Unfortunately code still not ready to be reviewed.
1) Code contains redundant comments.
for example: // refreshPartitions();
2) Redindant newlines should be removed. At code and imports.
3) Newlines should be added where it necessary:
for example:
Map res = process(tuple);
collector.emit(new Values(res));
4) Some methods have no javadoc.
5) @Override should be located according to Coding Guidelines.
6) Javadoc sentences should be finished with dot.
7) Git conflicts should be resolved, see GridSqlQueryParser.java for details.
Please recheck whole code matches Coding Guidelines and contains only files
related to this issue.
> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
> Issue Type: Sub-task
> Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under
> high load. It was previously named {{IgniteDataLoader}}, see ticket
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm
> and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm
[
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14909691#comment-14909691
]
Gianfranco Murador edited comment on IGNITE-429 at 9/27/15 2:19 PM:
The test was successful with the latest changes . The operation of the stream
is different compared to the one described in the guidelines because the
integration is via a Storm Bolt . I'd like to ask to Chandresh to provide a
simple documentation about this topic so that we can include it in official dev
wiki .
was (Author: glutters):
The test was successful with the latest changes . The operation of the stream
is different compared to the one described in the guidelines because the
integration is via a Storm Bolt . I ask to Chandresh to provide a simple
documentation about this topic so that we can include it in official dev wiki .
> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
> Issue Type: Sub-task
> Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under
> high load. It was previously named {{IgniteDataLoader}}, see ticket
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm
> and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm
[
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14909691#comment-14909691
]
Gianfranco Murador edited comment on IGNITE-429 at 9/27/15 12:53 PM:
-
The test was successful with the latest changes . The operation of the stream
is different compared to the one described in the guidelines because the
integration is via a Storm Bolt . I ask to Chandresh to provide a simple
documentation about this topic so that we can include it in official dev wiki .
was (Author: glutters):
The test was successful with the latest changes . The operation of the stream
is different compared to the one described in the guidelines because the
integration is via a Storm Bolt . I ask Chandresh provide documentation about
so that we can include in wiki .
> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
> Issue Type: Sub-task
> Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under
> high load. It was previously named {{IgniteDataLoader}}, see ticket
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm
> and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-429) Implement IgniteStormStreamer to stream data from Apache Storm
[
https://issues.apache.org/jira/browse/IGNITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877480#comment-14877480
]
Gianfranco Murador edited comment on IGNITE-429 at 9/20/15 11:51 AM:
-
Hello Chandresh,
the implementation is simple ( this is good ), but It does not take into
account of some aspects that may not work in Ignite. As we have done in Kafka
streamer is a good idea to create an ExecutorService to submit streams from
apache storm (This improves performance) . You have to rely to a naming
convention that must be declared with your solution, while it might be a good
idea to create our bolt to filter results. I will make a new branch with all
the changes, so you can make a new pull request. Still I am not very skilled
with git :).
Regards, Gianfranco
was (Author: glutters):
Hello Chandresh,
how do you see the implementation is simple ( this is good ), but It does not
take into account of some aspects that may not work in Ignite. As we have done
in Kafka streamer is a good idea to create an ExecutorService to submit streams
from apache storm (This improves performance) . You have to rely to a naming
convention that must be declared with your solution, while it might be a good
idea to create our bolt to filter results. I will make a new branch with all
the changes, so you can make a new pull request. Still I am not very skilled
with git :).
Regards, Gianfranco
> Implement IgniteStormStreamer to stream data from Apache Storm
> --
>
> Key: IGNITE-429
> URL: https://issues.apache.org/jira/browse/IGNITE-429
> Project: Ignite
> Issue Type: Sub-task
> Components: streaming
>Reporter: Dmitriy Setrakyan
>Assignee: Chandresh Pancholi
>
> We have {{IgniteDataStreamer}} which is used to load data into Ignite under
> high load. It was previously named {{IgniteDataLoader}}, see ticket
> IGNITE-394.
> See [Apache Storm|https://storm.apache.org/] for more information.
> We should create {{IgniteStormStreamer}} which will consume tuples from Storm
> and stream them into Ignite caches.
> More details to follow, but to the least we should be able to:
> - Convert/Intercept Storm tuples to Ignite data using an optional pluggable
> converter.
> - Specify the cache name for the Ignite cache to load data into.
> - Specify other flags available on {{IgniteDataStreamer}} class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
