[
https://issues.apache.org/jira/browse/STORM-871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621619#comment-14621619
]
ASF GitHub Bot commented on STORM-871:
--------------------------------------
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/602#discussion_r34323522
--- Diff: storm-multilang/python/src/main/resources/resources/storm.py ---
@@ -29,6 +30,26 @@
json_encode = lambda x: json.dumps(x)
json_decode = lambda x: json.loads(x)
+# scheduled timer running job at fixed rate (note: GIL!)
--- End diff --
Thanks @dan-blanchard ,
I experimented about python's GIL just now, and with python 2.7.6 in OSX I
found that other thread can hold CPU more than 1 sec when timer is expired at
that time.
https://gist.github.com/HeartSaVioR/34d90cdd6af906e72935
Actually I wasn't affected this issue during I was working with Python
cause it was I/O intensive job, and seems like it isn't same to CPU intensive
job.
Default tick time is somewhat very long. I found one document which says
tick time is about ~6.5 secs, which doesn't meet our requirement.
GIL is a major problem, and maybe I should check it with Ruby.
> Change multilang heartbeat mechanism to multi-threading of subprocess
> ---------------------------------------------------------------------
>
> Key: STORM-871
> URL: https://issues.apache.org/jira/browse/STORM-871
> Project: Apache Storm
> Issue Type: Improvement
> Reporter: Jungtaek Lim
> Assignee: Jungtaek Lim
>
> Having design constraint of multilang heartbeat, some issues are raised at
> the moment.
> (https://issues.apache.org/jira/browse/STORM-738 is a kind of reporting)
> I'm trying to add workarounds
> (https://issues.apache.org/jira/browse/STORM-742 and so on), but if we can
> get rid of design constraint it would be really great.
> Maybe we can use pid file of subprocess (subprocess write it from sendpid())
> to check subprocess heartbeat from ShellSpout / ShellBolt.
> Letting subprocess launch new thread which writes pid file periodically, to
> make sure "modified time" of file is periodically up to date. (It is just an
> idea, and it should be validated.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
