[
https://issues.apache.org/jira/browse/STORM-871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620963#comment-14620963
]
ASF GitHub Bot commented on STORM-871:
--------------------------------------
Github user dan-blanchard commented on a diff in the pull request:
https://github.com/apache/storm/pull/602#discussion_r34284915
--- Diff: storm-multilang/python/src/main/resources/resources/storm.py ---
@@ -29,6 +30,26 @@
json_encode = lambda x: json.dumps(x)
json_decode = lambda x: json.loads(x)
+# scheduled timer running job at fixed rate (note: GIL!)
--- End diff --
As you note, the GIL is going to be a major problem with this approach. To
the point, that this will not work as intended and you will still have timeout
issues if processing an individual tuple takes too long.
> Change multilang heartbeat mechanism to multi-threading of subprocess
> ---------------------------------------------------------------------
>
> Key: STORM-871
> URL: https://issues.apache.org/jira/browse/STORM-871
> Project: Apache Storm
> Issue Type: Improvement
> Reporter: Jungtaek Lim
> Assignee: Jungtaek Lim
>
> Having design constraint of multilang heartbeat, some issues are raised at
> the moment.
> (https://issues.apache.org/jira/browse/STORM-738 is a kind of reporting)
> I'm trying to add workarounds
> (https://issues.apache.org/jira/browse/STORM-742 and so on), but if we can
> get rid of design constraint it would be really great.
> Maybe we can use pid file of subprocess (subprocess write it from sendpid())
> to check subprocess heartbeat from ShellSpout / ShellBolt.
> Letting subprocess launch new thread which writes pid file periodically, to
> make sure "modified time" of file is periodically up to date. (It is just an
> idea, and it should be validated.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
