[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096608#comment-15096608 ] ASF GitHub Bot commented on STORM-1419: --- Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/977 > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096352#comment-15096352 ] ASF GitHub Bot commented on STORM-1419: --- Github user hmcl commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-171332402 +1 LGTM > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096322#comment-15096322 ] ASF GitHub Bot commented on STORM-1419: --- Github user revans2 commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-171325985 +1 > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095719#comment-15095719 ] ASF GitHub Bot commented on STORM-1419: --- Github user vesense commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-171195222 @revans2 @hmcl Conflicts resolved. > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15092132#comment-15092132 ] ASF GitHub Bot commented on STORM-1419: --- Github user revans2 commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-170588715 +1, but the code looks like it needs to be upmerged. > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15091125#comment-15091125 ] ASF GitHub Bot commented on STORM-1419: --- Github user hmcl commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-170373919 +1 LGTM @vesense to solve the conflicts you can rebase > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15090969#comment-15090969
]
ASF GitHub Bot commented on STORM-1419:
---
Github user vesense commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49274112
--- Diff: storm-core/src/jvm/backtype/storm/utils/TupleUtils.java ---
@@ -28,8 +35,21 @@ private TupleUtils() {
public static boolean isTick(Tuple tuple) {
return tuple != null
- && Constants.SYSTEM_COMPONENT_ID
.equals(tuple.getSourceComponent())
+ &&
Constants.SYSTEM_COMPONENT_ID.equals(tuple.getSourceComponent())
&&
Constants.SYSTEM_TICK_STREAM_ID.equals(tuple.getSourceStreamId());
}
+ public static Map
createComponentConfigByTickFreqSecs(Map conf, int tickFreqSecs)
{
--- End diff --
Done.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15090968#comment-15090968
]
ASF GitHub Bot commented on STORM-1419:
---
Github user vesense commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49274109
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -59,16 +63,29 @@ public SolrUpdateBolt(SolrConfig solrConfig, SolrMapper
solrMapper, SolrCommitSt
this.solrConfig = solrConfig;
this.solrMapper = solrMapper;
this.commitStgy = commitStgy;
-logger.debug("Created {} with the following configuration: " +
+LOG.debug("Created {} with the following configuration: " +
"[SolrConfig = {}], [SolrMapper = {}], [CommitStgy =
{}]",
this.getClass().getSimpleName(), solrConfig,
solrMapper, commitStgy);
}
+@Override
public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
this.collector = collector;
this.solrClient = new
CloudSolrClient(solrConfig.getZkHostString());
this.toCommitTuples = new ArrayList<>(capacity());
+this.tickTupleInterval = solrConfig.getTickTupleInterval();
--- End diff --
Done.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15090759#comment-15090759
]
ASF GitHub Bot commented on STORM-1419:
---
Github user hmcl commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49267676
--- Diff: storm-core/src/jvm/backtype/storm/utils/TupleUtils.java ---
@@ -28,8 +35,21 @@ private TupleUtils() {
public static boolean isTick(Tuple tuple) {
return tuple != null
- && Constants.SYSTEM_COMPONENT_ID
.equals(tuple.getSourceComponent())
+ &&
Constants.SYSTEM_COMPONENT_ID.equals(tuple.getSourceComponent())
&&
Constants.SYSTEM_TICK_STREAM_ID.equals(tuple.getSourceStreamId());
}
+ public static Map
createComponentConfigByTickFreqSecs(Map conf, int tickFreqSecs)
{
--- End diff --
@vesense I would rename this method to 'putTickFreqencyIntoComponentConfig'
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15090733#comment-15090733
]
ASF GitHub Bot commented on STORM-1419:
---
Github user hmcl commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49266446
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -59,16 +63,29 @@ public SolrUpdateBolt(SolrConfig solrConfig, SolrMapper
solrMapper, SolrCommitSt
this.solrConfig = solrConfig;
this.solrMapper = solrMapper;
this.commitStgy = commitStgy;
-logger.debug("Created {} with the following configuration: " +
+LOG.debug("Created {} with the following configuration: " +
"[SolrConfig = {}], [SolrMapper = {}], [CommitStgy =
{}]",
this.getClass().getSimpleName(), solrConfig,
solrMapper, commitStgy);
}
+@Override
public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
this.collector = collector;
this.solrClient = new
CloudSolrClient(solrConfig.getZkHostString());
this.toCommitTuples = new ArrayList<>(capacity());
+this.tickTupleInterval = solrConfig.getTickTupleInterval();
--- End diff --
@vesense I would suggest that all the code handling the tickTupleInterval
goes inside the method setTickTupleInterval. It is cleaner then setting the
field tickTupleInterval in prepare, and then have an if, and then set the field
again in setTickTupleInterval. The default case should be handled there as well.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15090668#comment-15090668 ] ASF GitHub Bot commented on STORM-1419: --- Github user vesense commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-170254404 @hmcl all comments addressed. but I met some conficts. > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15090639#comment-15090639
]
ASF GitHub Bot commented on STORM-1419:
---
Github user vesense commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49264396
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/config/SolrConfig.java
---
@@ -39,4 +40,13 @@ public SolrConfig(String zkHostString) {
public String getZkHostString() {
return zkHostString;
}
+
+public int getTickTupleInterval() {
+return tickTupleInterval;
+}
+
+public void setTickTupleInterval(int tickTupleInterval) {
--- End diff --
OK
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15090637#comment-15090637
]
ASF GitHub Bot commented on STORM-1419:
---
Github user vesense commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49264392
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -131,6 +150,21 @@ private void failQueuedTuples(List
failedTuples) {
return queuedTuples;
}
+@Override
+public Map getComponentConfiguration() {
--- End diff --
OK. The helper method is `createComponentConfigByTickFreqSecs` in
TupleUtils.
And I will make a new PR for other external modules.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15090638#comment-15090638
]
ASF GitHub Bot commented on STORM-1419:
---
Github user vesense commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49264394
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -59,15 +63,24 @@ public SolrUpdateBolt(SolrConfig solrConfig, SolrMapper
solrMapper, SolrCommitSt
this.solrConfig = solrConfig;
this.solrMapper = solrMapper;
this.commitStgy = commitStgy;
-logger.debug("Created {} with the following configuration: " +
+LOG.debug("Created {} with the following configuration: " +
"[SolrConfig = {}], [SolrMapper = {}], [CommitStgy =
{}]",
this.getClass().getSimpleName(), solrConfig,
solrMapper, commitStgy);
}
+@Override
public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
this.collector = collector;
this.solrClient = new
CloudSolrClient(solrConfig.getZkHostString());
this.toCommitTuples = new ArrayList<>(capacity());
+this.tickTupleInterval = solrConfig.getTickTupleInterval();
+
+//set default tickTupleInterval
+if (stormConf.containsKey("topology.message.timeout.secs") &&
tickTupleInterval == 0) {
--- End diff --
OK
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15090636#comment-15090636
]
ASF GitHub Bot commented on STORM-1419:
---
Github user vesense commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49264374
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -92,9 +108,12 @@ private void ack(Tuple tuple) throws
SolrServerException, IOException {
if (commitStgy == null) {
collector.ack(tuple);
} else {
-toCommitTuples.add(tuple);
-commitStgy.update();
-if (commitStgy.commit()) {
+final boolean isTickTuple = TupleUtils.isTick(tuple);
+if (!isTickTuple) {
--- End diff --
OK
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088769#comment-15088769 ] ASF GitHub Bot commented on STORM-1419: --- Github user hmcl commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-169908195 +1 once the last few, minor, comments are addressed. @vesense Thanks for the great work! > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088767#comment-15088767
]
ASF GitHub Bot commented on STORM-1419:
---
Github user hmcl commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49161066
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/config/SolrConfig.java
---
@@ -39,4 +40,13 @@ public SolrConfig(String zkHostString) {
public String getZkHostString() {
return zkHostString;
}
+
+public int getTickTupleInterval() {
+return tickTupleInterval;
+}
+
+public void setTickTupleInterval(int tickTupleInterval) {
--- End diff --
@vesense Can you please delete this set method and pass the
tickTupleInterval in the constructor, or use a builder pattern. This object
should be immutable and fully initialized once passed to the Bolt.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088765#comment-15088765
]
ASF GitHub Bot commented on STORM-1419:
---
Github user hmcl commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49160962
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -59,15 +63,24 @@ public SolrUpdateBolt(SolrConfig solrConfig, SolrMapper
solrMapper, SolrCommitSt
this.solrConfig = solrConfig;
this.solrMapper = solrMapper;
this.commitStgy = commitStgy;
-logger.debug("Created {} with the following configuration: " +
+LOG.debug("Created {} with the following configuration: " +
"[SolrConfig = {}], [SolrMapper = {}], [CommitStgy =
{}]",
this.getClass().getSimpleName(), solrConfig,
solrMapper, commitStgy);
}
+@Override
public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
this.collector = collector;
this.solrClient = new
CloudSolrClient(solrConfig.getZkHostString());
this.toCommitTuples = new ArrayList<>(capacity());
+this.tickTupleInterval = solrConfig.getTickTupleInterval();
+
+//set default tickTupleInterval
+if (stormConf.containsKey("topology.message.timeout.secs") &&
tickTupleInterval == 0) {
--- End diff --
@vesense Please create a private method called something
setDefaultTickTupleInterval() with this code and call it here.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088764#comment-15088764
]
ASF GitHub Bot commented on STORM-1419:
---
Github user hmcl commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49160913
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -131,6 +150,21 @@ private void failQueuedTuples(List
failedTuples) {
return queuedTuples;
}
+@Override
+public Map getComponentConfiguration() {
--- End diff --
@vesense I noticed that the body of this method is exactly the same for
AbstractHdfsBolt. Furthermore, it's very likely that many Storm connector bolts
will use the same code. Can you please write a helper method in TupleUtils and
call that method here and AbstractHdfsBolt.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088760#comment-15088760
]
ASF GitHub Bot commented on STORM-1419:
---
Github user hmcl commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49160816
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -92,9 +108,12 @@ private void ack(Tuple tuple) throws
SolrServerException, IOException {
if (commitStgy == null) {
collector.ack(tuple);
} else {
-toCommitTuples.add(tuple);
-commitStgy.update();
-if (commitStgy.commit()) {
+final boolean isTickTuple = TupleUtils.isTick(tuple);
+if (!isTickTuple) {
--- End diff --
Can you please add the inline comment: // Don't ack tick tuples.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088712#comment-15088712 ] ASF GitHub Bot commented on STORM-1419: --- Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-169896957 +1 > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088696#comment-15088696 ] ASF GitHub Bot commented on STORM-1419: --- Github user vesense commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-169894466 @hmcl comments addressed. > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088687#comment-15088687
]
ASF GitHub Bot commented on STORM-1419:
---
Github user vesense commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49157386
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -59,16 +64,24 @@ public SolrUpdateBolt(SolrConfig solrConfig, SolrMapper
solrMapper, SolrCommitSt
this.solrConfig = solrConfig;
this.solrMapper = solrMapper;
this.commitStgy = commitStgy;
-logger.debug("Created {} with the following configuration: " +
+LOG.debug("Created {} with the following configuration: " +
"[SolrConfig = {}], [SolrMapper = {}], [CommitStgy =
{}]",
this.getClass().getSimpleName(), solrConfig,
solrMapper, commitStgy);
}
+@Override
public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
this.collector = collector;
this.solrClient = new
CloudSolrClient(solrConfig.getZkHostString());
this.toCommitTuples = new ArrayList<>(capacity());
+//set default tickTupleInterval if interval is zero
+if (stormConf.containsKey("topology.message.timeout.secs") &&
tickTupleInterval == 0) {
--- End diff --
I prefer to keep this code in `prepare()` method since no `stormConf` in
SolrConfig.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088678#comment-15088678
]
ASF GitHub Bot commented on STORM-1419:
---
Github user vesense commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49157203
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -92,9 +108,17 @@ private void ack(Tuple tuple) throws
SolrServerException, IOException {
if (commitStgy == null) {
collector.ack(tuple);
} else {
-toCommitTuples.add(tuple);
-commitStgy.update();
-if (commitStgy.commit()) {
+boolean forceCommit = false;
+if (TupleUtils.isTick(tuple)) {
+LOG.debug("TICK! forcing solr client commit");
+collector.ack(tuple);
+forceCommit = true;
+} else {
+toCommitTuples.add(tuple);
+commitStgy.update();
+}
+
+if (forceCommit || commitStgy.commit()) {
--- End diff --
OK
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088679#comment-15088679
]
ASF GitHub Bot commented on STORM-1419:
---
Github user vesense commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49157210
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -92,9 +108,17 @@ private void ack(Tuple tuple) throws
SolrServerException, IOException {
if (commitStgy == null) {
collector.ack(tuple);
} else {
-toCommitTuples.add(tuple);
-commitStgy.update();
-if (commitStgy.commit()) {
+boolean forceCommit = false;
+if (TupleUtils.isTick(tuple)) {
+LOG.debug("TICK! forcing solr client commit");
+collector.ack(tuple);
--- End diff --
OK
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088680#comment-15088680
]
ASF GitHub Bot commented on STORM-1419:
---
Github user vesense commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49157240
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -131,6 +155,26 @@ private void failQueuedTuples(List
failedTuples) {
return queuedTuples;
}
+public SolrUpdateBolt withTickIntervalSecs(int tickTupleInterval) {
--- End diff --
OK. Thx.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088321#comment-15088321
]
ASF GitHub Bot commented on STORM-1419:
---
Github user hmcl commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49138053
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -59,16 +64,24 @@ public SolrUpdateBolt(SolrConfig solrConfig, SolrMapper
solrMapper, SolrCommitSt
this.solrConfig = solrConfig;
this.solrMapper = solrMapper;
this.commitStgy = commitStgy;
-logger.debug("Created {} with the following configuration: " +
+LOG.debug("Created {} with the following configuration: " +
"[SolrConfig = {}], [SolrMapper = {}], [CommitStgy =
{}]",
this.getClass().getSimpleName(), solrConfig,
solrMapper, commitStgy);
}
+@Override
public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
this.collector = collector;
this.solrClient = new
CloudSolrClient(solrConfig.getZkHostString());
this.toCommitTuples = new ArrayList<>(capacity());
+//set default tickTupleInterval if interval is zero
+if (stormConf.containsKey("topology.message.timeout.secs") &&
tickTupleInterval == 0) {
--- End diff --
@vesense This code should go in the SolrConfig object
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088297#comment-15088297
]
ASF GitHub Bot commented on STORM-1419:
---
Github user hmcl commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49136662
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -131,6 +155,26 @@ private void failQueuedTuples(List
failedTuples) {
return queuedTuples;
}
+public SolrUpdateBolt withTickIntervalSecs(int tickTupleInterval) {
--- End diff --
@vesense this violates encapsulation and should be avoided
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088292#comment-15088292
]
ASF GitHub Bot commented on STORM-1419:
---
Github user hmcl commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49136481
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -92,9 +108,17 @@ private void ack(Tuple tuple) throws
SolrServerException, IOException {
if (commitStgy == null) {
collector.ack(tuple);
} else {
-toCommitTuples.add(tuple);
-commitStgy.update();
-if (commitStgy.commit()) {
+boolean forceCommit = false;
+if (TupleUtils.isTick(tuple)) {
+LOG.debug("TICK! forcing solr client commit");
+collector.ack(tuple);
--- End diff --
tick tuples don't need to be acked
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088291#comment-15088291
]
ASF GitHub Bot commented on STORM-1419:
---
Github user hmcl commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r49136454
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -92,9 +108,17 @@ private void ack(Tuple tuple) throws
SolrServerException, IOException {
if (commitStgy == null) {
collector.ack(tuple);
} else {
-toCommitTuples.add(tuple);
-commitStgy.update();
-if (commitStgy.commit()) {
+boolean forceCommit = false;
+if (TupleUtils.isTick(tuple)) {
+LOG.debug("TICK! forcing solr client commit");
+collector.ack(tuple);
+forceCommit = true;
+} else {
+toCommitTuples.add(tuple);
+commitStgy.update();
+}
+
+if (forceCommit || commitStgy.commit()) {
--- End diff --
@vesense Can you please follow te suggestion that I gave as the code is
much cleaner.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086184#comment-15086184 ] ASF GitHub Bot commented on STORM-1419: --- Github user hmcl commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-169441633 @revans2 your points are valid, as there may be scenarios where we may end up committing very small batches, which may cause some performance degradation, but in my opinion that would only be a concern if they are likely to happen. Nevertheless, to address your points I have created https://issues.apache.org/jira/browse/STORM-1442 Concerning your comment " It looks like the commitStgy.commit() is not needed at all. " I believe we still need it to address the cases where the batch is complete, no tick tuples came in, and we want to commit the batch. > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086174#comment-15086174 ] ASF GitHub Bot commented on STORM-1419: --- Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-169440883 What @hmcl is suggesting also fixes the bug that trying to convert tick tuple to Solr request and do actual request. Great finding. > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086166#comment-15086166 ] ASF GitHub Bot commented on STORM-1419: --- Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-169439287 @hmcl You're right. My bad. :) > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086162#comment-15086162 ] ASF GitHub Bot commented on STORM-1419: --- Github user hmcl commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-169438537 @HeartSaVioR All the tuples that have already been batched will still need to be acked, therefore we cannot simply ignore the tick tuple. All I am doing is that when calling ack() I don't add the tick tuple to the queue of tuples that need to be acked. > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086153#comment-15086153 ] ASF GitHub Bot commented on STORM-1419: --- Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-169437074 @hmcl If we think we don't need to ack tick tuple, ignoring tick tuple from execute() first would be more clearer. Then we don't need to take care of tick tuple from ack(). > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086150#comment-15086150 ] ASF GitHub Bot commented on STORM-1419: --- Github user hmcl commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-169436285 I would like to suggest that we address the tick tuples as follows:  > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15085561#comment-15085561 ] ASF GitHub Bot commented on STORM-1419: --- Github user dossett commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-169330085 @vesense Thanks for the info about the global configuration. Aside from the issues @HeartSaVioR brings up, I think adding bolt-level option for tick tuples to the solr bolt is necessary. Without that, the only way for a user to enable ticks for the solr bolt is to enable them for the entire topology. That's very different than the other output bolts (hdfs, hive) that support ticks. > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15085209#comment-15085209 ] ASF GitHub Bot commented on STORM-1419: --- Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-169267211 @vesense Please read https://github.com/apache/storm/pull/977#issuecomment-169101930 and https://github.com/apache/storm/pull/977#issuecomment-169146734. Easily thinkable edge case is, Solr can commit pending requests internally, but SolrUpdateBolt still has tuples in pending state for ack. It means we should always set tick tuple to commit externally. > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15085194#comment-15085194
]
ASF GitHub Bot commented on STORM-1419:
---
Github user vesense commented on the pull request:
https://github.com/apache/storm/pull/977#issuecomment-169265399
Respond to @dossett comments:
> I am not familiar with all the ways that bolts can receive tick tuples,
but does SolrBolt also need a way to specify that it should receive tick tuples
at all?
To my knowledge, there are two ways that bolts can receive tick tuples. One
is global setting when building topology.
```
Config config = new Config();
config.put(Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS, 5);
```
The other one is local setting in bolt.
```
@Override
public Map getComponentConfiguration() {
Map conf = new HashMap();
conf.put(Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS, 4);
return conf;
}
```
All bolt tasks will receive tick tuples in the same interval when existing
global setting.
Once we set `Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS` in bolt Foo
`getComponentConfiguration()`, the value will override the global setting.
And bolt Foo tasks will receive tick tuples in a interval different with
other tasks.
So, it is not necessary to add `Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS` in
`getComponentConfiguration()`. It depends on the user scenario.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15085141#comment-15085141
]
ASF GitHub Bot commented on STORM-1419:
---
Github user harshach commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r48933699
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -92,11 +94,19 @@ private void ack(Tuple tuple) throws
SolrServerException, IOException {
if (commitStgy == null) {
collector.ack(tuple);
} else {
-toCommitTuples.add(tuple);
-commitStgy.update();
-if (commitStgy.commit()) {
+if (TupleUtils.isTick(tuple)) {
+LOG.debug("TICK! forcing solr client commit");
+collector.ack(tuple);
--- End diff --
no need for tick tuples to be acked. We've similar thing in storm-hive and
we don't ack tick tuples.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15085112#comment-15085112 ] ASF GitHub Bot commented on STORM-1419: --- Github user harshach commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-169256465 @hmcl can you review this. > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15085076#comment-15085076
]
ASF GitHub Bot commented on STORM-1419:
---
Github user satishd commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r48931925
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -92,11 +94,19 @@ private void ack(Tuple tuple) throws
SolrServerException, IOException {
if (commitStgy == null) {
collector.ack(tuple);
} else {
-toCommitTuples.add(tuple);
-commitStgy.update();
-if (commitStgy.commit()) {
+if (TupleUtils.isTick(tuple)) {
+LOG.debug("TICK! forcing solr client commit");
+collector.ack(tuple);
--- End diff --
I am also not sure why tick-tuples need to be acked as @arunmahadevan
mentioned. Whenever an executor is started it schedules a timer which sends
tick tuples to receive queue
``` clojure
(defn setup-ticks! [worker executor-data]
(let [storm-conf (:storm-conf executor-data)
tick-time-secs (storm-conf TOPOLOGY-TICK-TUPLE-FREQ-SECS)
receive-queue (:receive-queue executor-data)
context (:worker-context executor-data)]
(when tick-time-secs
(if (or (Utils/isSystemId (:component-id executor-data))
(and (= false (storm-conf TOPOLOGY-ENABLE-MESSAGE-TIMEOUTS))
(= :spout (:type executor-data
(log-message "Timeouts disabled for executor " (:component-id
executor-data) ":" (:executor-id executor-data))
(schedule-recurring
(:user-timer worker)
tick-time-secs
tick-time-secs
(fn []
(let [val [(AddressedTuple. AddressedTuple/BROADCAST_DEST
(TupleImpl. context [tick-time-secs] Constants/SYSTEM_TASK_ID
Constants/SYSTEM_TICK_STREAM_ID))]]
(disruptor/publish receive-queue val
```
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15084973#comment-15084973
]
ASF GitHub Bot commented on STORM-1419:
---
Github user arunmahadevan commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r48927497
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -92,11 +94,19 @@ private void ack(Tuple tuple) throws
SolrServerException, IOException {
if (commitStgy == null) {
collector.ack(tuple);
} else {
-toCommitTuples.add(tuple);
-commitStgy.update();
-if (commitStgy.commit()) {
+if (TupleUtils.isTick(tuple)) {
+LOG.debug("TICK! forcing solr client commit");
+collector.ack(tuple);
--- End diff --
Tick tuples are not generated from a spout but inserted periodically into
the executor receive-queue. Since tick-tuples are not tracked by the acker, I
dont think it needs to be acked. Acking it does not cause any harm but
unnecessary. Correct me if I am overlooking something.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083939#comment-15083939 ] ASF GitHub Bot commented on STORM-1419: --- Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-169151251 @vesense Could you handle what @dossett stated? > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083937#comment-15083937 ] ASF GitHub Bot commented on STORM-1419: --- Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-169150861 @dossett Yes, you're right. SolrUpdateBolt should force setting up tick tuple since without tick tuple we may get odd situation @revans2 and I stated. > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083920#comment-15083920 ] ASF GitHub Bot commented on STORM-1419: --- Github user dossett commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-169147171 I am not familiar with all the ways that bolts can receive tick tuples, but does SolrBolt also need a way to specify that it should receive tick tuples at all? The hdfs-bolt way of doing that is here: https://github.com/apache/storm/blob/master/external/storm-hdfs/src/main/java/org/apache/storm/hdfs/bolt/AbstractHdfsBolt.java#L221 > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083916#comment-15083916 ] ASF GitHub Bot commented on STORM-1419: --- Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-169146734 I am not an expert on the Solr code, and Solr itself. So below comment may need to have corrections via other contributors / committers. Looking into SolrCommitStrategy and CountBasedCommit I feel the same thing with @revans2. State for the SolrCommitStrategy could be out of sync because it cannot know `internal commit` (not requested via commit()) yet, and looking into javadoc for SolrClient, there's no way to check internal commit. Here's description for rollback, which describes the internal commit. https://lucene.apache.org/solr/5_2_0/solr-solrj/org/apache/solr/client/solrj/SolrClient.html#rollback() > rollback > >Performs a rollback of all non-committed documents pending. Note that this is not a true rollback as in databases. Content you have previously added may have been committed due to autoCommit, buffer full, other client performing a commit etc. Based on internal commit, I see current code has an issue, which could be minor, or maybe major. > There could be some requests which are committed internally via Solr but not acked / failed via Bolt's side. (It will be resolved when SolrUpdateBolt deals with tick tuple.) It means acknowledges of some tuples can be lost though tuples are committed to Solr. If we intend to let requests buffered and in perfect sync with ack / fail, I think it shouldn't request for each execute() but just stores requests and do actual requests / external commit to Solr / ack & fail when commit condition has been made. Or if we don't need to be accurate, I'm fine with the code change. > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083783#comment-15083783
]
ASF GitHub Bot commented on STORM-1419:
---
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r48895950
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -92,11 +94,19 @@ private void ack(Tuple tuple) throws
SolrServerException, IOException {
if (commitStgy == null) {
collector.ack(tuple);
} else {
-toCommitTuples.add(tuple);
-commitStgy.update();
-if (commitStgy.commit()) {
+if (TupleUtils.isTick(tuple)) {
+LOG.debug("TICK! forcing solr client commit");
+collector.ack(tuple);
+commitStgy.commit();
--- End diff --
Sorry I didn't check code deeply.
SolrCommitStrategy.commit() just checks whether it is time to commit or
not, so in this case it shouldn't check return value.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083587#comment-15083587
]
ASF GitHub Bot commented on STORM-1419:
---
Github user revans2 commented on the pull request:
https://github.com/apache/storm/pull/977#issuecomment-169101930
The code changes looks OK. I am not an expert on the Solr code, but I do
have a few concerns.
In response to comments from @HeartSaVioR and @dossett. It looks like the
`commitStgy.commit()` is not needed at all. That method appears to return a
true if a commit should happen. We are forcing a commit with the tick tuple so
calling it and checking the response is not needed. Which brings up my
concern. There is no documentation in CommitStrategy about how it is intended
to be used. Which methods are called when, etc. Without them it is hard to
tell if we are violating some type of contract by making this change. For now
I think this is OK, but it would be good to add in that documentation, and I
would love it to see another method added to the commit strategy (perhaps
beyond the scope of this code). That indicates a commit happened. So it would
look something like.
```
/**
* Strategy defining when the Solr Bolt should commit the request batch to
Solr.
*/
public interface SolrCommitStrategy extends Serializable {
/**
* @return true of a commit should happen.
*/
boolean shouldCommit();
/**
* Indicates that a commit occurred. This may happen even if
shouldCommit returned false, or was not even called at all to indicate that a
commit was forced.
*/
void commitHappened();
/**
* Indicates that a tuple was added to the current batch.
*/
void update();
}
```
The current count code has issues because it does not know that the commit
happened, so it uses a modulo with a check for 0, which means we could get
really odd situations where a commit was forced because of a timeout, but the
next batch only has a single entry in it.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083385#comment-15083385
]
ASF GitHub Bot commented on STORM-1419:
---
Github user ptgoetz commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r48869637
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -92,11 +94,19 @@ private void ack(Tuple tuple) throws
SolrServerException, IOException {
if (commitStgy == null) {
collector.ack(tuple);
} else {
-toCommitTuples.add(tuple);
-commitStgy.update();
-if (commitStgy.commit()) {
+if (TupleUtils.isTick(tuple)) {
+LOG.debug("TICK! forcing solr client commit");
+collector.ack(tuple);
--- End diff --
Yes, tick tuples should be ack'd.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083280#comment-15083280
]
ASF GitHub Bot commented on STORM-1419:
---
Github user dossett commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r48859729
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -92,11 +94,19 @@ private void ack(Tuple tuple) throws
SolrServerException, IOException {
if (commitStgy == null) {
collector.ack(tuple);
} else {
-toCommitTuples.add(tuple);
-commitStgy.update();
-if (commitStgy.commit()) {
+if (TupleUtils.isTick(tuple)) {
+LOG.debug("TICK! forcing solr client commit");
+collector.ack(tuple);
+commitStgy.commit();
solrClient.commit(solrMapper.getCollection());
ackCommittedTuples();
+} else {
+toCommitTuples.add(tuple);
+commitStgy.update();
+if (commitStgy.commit()) {
--- End diff --
The strategy in AbstractHdfsBolt is to set a boolean in the case of a tick
tuple and then sync if that value is true or if other conditions dictate a
sync.
(https://github.com/apache/storm/blob/master/external/storm-hdfs/src/main/java/org/apache/storm/hdfs/bolt/AbstractHdfsBolt.java#L154)
The benefit of that approach is to eliminate duplicate code (i.e. that
calls to ackCommittedTuples() and solrClient.commit()), which I think is a
substantial benefit.
Here that would looks something like:
```code
if (forceCommit || commitStgy.commit()) {
solrClient.commit(solrMapper.getCollection());
ackCommittedTuples();
}
```
With duplicate code removed I would be +1
A unit test would also be helpful. HdfsBolt example is here:
https://github.com/apache/storm/blob/master/external/storm-hdfs/src/test/java/org/apache/storm/hdfs/bolt/TestHdfsBolt.java#L175
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083279#comment-15083279
]
ASF GitHub Bot commented on STORM-1419:
---
Github user dossett commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r48859713
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -92,11 +94,19 @@ private void ack(Tuple tuple) throws
SolrServerException, IOException {
if (commitStgy == null) {
collector.ack(tuple);
} else {
-toCommitTuples.add(tuple);
-commitStgy.update();
-if (commitStgy.commit()) {
+if (TupleUtils.isTick(tuple)) {
+LOG.debug("TICK! forcing solr client commit");
+collector.ack(tuple);
--- End diff --
Should tick tuples be ack'd? I believe that is an open question in another
PR.
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[
https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082127#comment-15082127
]
ASF GitHub Bot commented on STORM-1419:
---
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/977#discussion_r48802421
--- Diff:
external/storm-solr/src/main/java/org/apache/storm/solr/bolt/SolrUpdateBolt.java
---
@@ -92,11 +94,19 @@ private void ack(Tuple tuple) throws
SolrServerException, IOException {
if (commitStgy == null) {
collector.ack(tuple);
} else {
-toCommitTuples.add(tuple);
-commitStgy.update();
-if (commitStgy.commit()) {
+if (TupleUtils.isTick(tuple)) {
+LOG.debug("TICK! forcing solr client commit");
+collector.ack(tuple);
+commitStgy.commit();
--- End diff --
Shouldn't we make it consistent with the else statement?
I mean, shouldn't we check the return value of commitStgy.commit() and do
other actions?
> Solr bolt should handle tick tuples
> ---
>
> Key: STORM-1419
> URL: https://issues.apache.org/jira/browse/STORM-1419
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-solr
>Reporter: Xin Wang
>Assignee: Xin Wang
>
> Solr bolt should handle tick tuples.
> Forcing solr client commit when bolt received tick tuple.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15081774#comment-15081774 ] ASF GitHub Bot commented on STORM-1419: --- Github user ptgoetz commented on the pull request: https://github.com/apache/storm/pull/977#issuecomment-168801983 +1 > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1419) Solr bolt should handle tick tuples
[ https://issues.apache.org/jira/browse/STORM-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15071497#comment-15071497 ] ASF GitHub Bot commented on STORM-1419: --- GitHub user vesense opened a pull request: https://github.com/apache/storm/pull/977 [STORM-1419] Solr bolt should handle tick tuples You can merge this pull request into a Git repository by running: $ git pull https://github.com/vesense/storm STORM-1419 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/977.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #977 commit 397733c054c3ba120bc6670f54e5d56398969180 Author: Xin Wang Date: 2015-12-25T10:20:42Z solr bolt should handle tick tuples > Solr bolt should handle tick tuples > --- > > Key: STORM-1419 > URL: https://issues.apache.org/jira/browse/STORM-1419 > Project: Apache Storm > Issue Type: Bug > Components: storm-solr >Reporter: Xin Wang >Assignee: Xin Wang > > Solr bolt should handle tick tuples. > Forcing solr client commit when bolt received tick tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
