[jira] [Commented] (STORM-2761) JoinBolt.java 's paradigm is new model of stream join?
[ https://issues.apache.org/jira/browse/STORM-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181944#comment-16181944 ] Arun Mahadevan commented on STORM-2761: --- As per my understanding the tuples are buffered in both streams and joined only once when the window triggers. e.g with a 1 min window all tuples that arrived in the last 1 min in "stream1" is joined with all the tuples that arrived in the last 1 min in "stream2" when the 1 min completes. If it does not work that way there might be a bug. cc [~roshan_naik] > JoinBolt.java 's paradigm is new model of stream join? > -- > > Key: STORM-2761 > URL: https://issues.apache.org/jira/browse/STORM-2761 > Project: Apache Storm > Issue Type: Question > Components: storm-client >Reporter: Fei Pan >Priority: Critical > > Hi, I am a researcher from University of Toronto and I am studying > acceleration on stream processing platform. I have a question about the model > of window-based stream join used in the JoinBolt.java. From my understanding, > when a new tuple arrived, we join this new tuple with all the tuples in the > window of the opposite stream. However, in the JoinBolt.java, not only the > new tuple, but the tuples in the entire local window will join with the > window of the opposite stream. This actually produces a lot of duplicated > results, since most of the old tuples in the local window have joined before. > I don't know if this is a new paradigm or the storm's team misunderstood the > model of stream join. Can someone help me to clarify this question? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (STORM-2762) Sort out multiple places of codes of manipulation of collections
Jungtaek Lim created STORM-2762: --- Summary: Sort out multiple places of codes of manipulation of collections Key: STORM-2762 URL: https://issues.apache.org/jira/browse/STORM-2762 Project: Apache Storm Issue Type: Bug Affects Versions: 2.0.0 Reporter: Jungtaek Lim Priority: Minor Storm has codes of manipulation of collections at multiple places. One of example would be blacklist/Sets.java which will be merged in via STORM-2083. https://github.com/apache/storm/pull/2343 It would be better to sort out and make them common utils, or if possible, replace them with specific library Storm already depends on. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (STORM-2743) Add logging to monitor how long scheduling is taking
[ https://issues.apache.org/jira/browse/STORM-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans resolved STORM-2743. Resolution: Fixed Fix Version/s: 2.0.0 Thanks [~ethanli], I merged this into master. > Add logging to monitor how long scheduling is taking > > > Key: STORM-2743 > URL: https://issues.apache.org/jira/browse/STORM-2743 > Project: Apache Storm > Issue Type: Improvement > Components: storm-server >Reporter: Ethan Li >Assignee: Ethan Li >Priority: Trivial > Labels: pull-request-available > Fix For: 2.0.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Add logging to monitor how long scheduling is taking -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (STORM-2757) Links are broken when logviewer https port is used
[
https://issues.apache.org/jira/browse/STORM-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated STORM-2757:
--
Labels: pull-request-available (was: )
> Links are broken when logviewer https port is used
> --
>
> Key: STORM-2757
> URL: https://issues.apache.org/jira/browse/STORM-2757
> Project: Apache Storm
> Issue Type: Bug
>Reporter: Ethan Li
>Assignee: Ethan Li
> Labels: pull-request-available
> Attachments: screenshot-1.png
>
>
> Some links are broken when logviewer.https.port is configured.
> For example, with the configuration:
> {code:java}
> logviewer.https.port: 9093
> logviewer.https.keystore.type: "JKS"
> logviewer.https.keystore.path: "/keystore-path"
> logviewer.https.keystore.password: "xx"
> logviewer.https.key.password: "xx"
> {code}
> We will get:
> [^screenshot-1.png]
> The logLink is still using http port. However, it's not reachable.
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (STORM-2761) JoinBolt.java 's paradigm is new model of stream join?
Fei Pan created STORM-2761: -- Summary: JoinBolt.java 's paradigm is new model of stream join? Key: STORM-2761 URL: https://issues.apache.org/jira/browse/STORM-2761 Project: Apache Storm Issue Type: Question Components: storm-client Reporter: Fei Pan Priority: Critical Hi, I am a researcher from University of Toronto and I am studying acceleration on stream processing platform. I have a question about the model of window-based stream join used in the JoinBolt.java. From my understanding, when a new tuple arrived, we join this new tuple with all the tuples in the window of the opposite stream. However, in the JoinBolt.java, not only the new tuple, but the tuples in the entire local window will join with the window of the opposite stream. This actually produces a lot of duplicated results, since most of the old tuples in the local window have joined before. I don't know if this is a new paradigm or the storm's team misunderstood the model of stream join. Can someone help me to clarify this question? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (STORM-2760) Add Blobstore Migration Scripts
[ https://issues.apache.org/jira/browse/STORM-2760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum updated STORM-2760: Description: Add code and helper scripts for migrating active Storm clusters from a locally-backed BlobStore to an HDFS-backed BlobStore. > Add Blobstore Migration Scripts > --- > > Key: STORM-2760 > URL: https://issues.apache.org/jira/browse/STORM-2760 > Project: Apache Storm > Issue Type: Improvement >Reporter: Kyle Nusbaum >Assignee: Kyle Nusbaum > Labels: pull-request-available > > Add code and helper scripts for migrating active Storm clusters from a > locally-backed BlobStore to an HDFS-backed BlobStore. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (STORM-2760) Add Blobstore Migration Scripts
[ https://issues.apache.org/jira/browse/STORM-2760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated STORM-2760: -- Labels: pull-request-available (was: ) > Add Blobstore Migration Scripts > --- > > Key: STORM-2760 > URL: https://issues.apache.org/jira/browse/STORM-2760 > Project: Apache Storm > Issue Type: Improvement >Reporter: Kyle Nusbaum >Assignee: Kyle Nusbaum > Labels: pull-request-available > > Add code and helper scripts for migrating active Storm clusters from a > locally-backed BlobStore to an HDFS-backed BlobStore. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (STORM-2760) Add Blobstore Migration Scripts
Kyle Nusbaum created STORM-2760: --- Summary: Add Blobstore Migration Scripts Key: STORM-2760 URL: https://issues.apache.org/jira/browse/STORM-2760 Project: Apache Storm Issue Type: Improvement Reporter: Kyle Nusbaum Assignee: Kyle Nusbaum -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (STORM-2759) Let users indicate if a worker should restart on blob download
Robert Joseph Evans created STORM-2759: -- Summary: Let users indicate if a worker should restart on blob download Key: STORM-2759 URL: https://issues.apache.org/jira/browse/STORM-2759 Project: Apache Storm Issue Type: Improvement Reporter: Robert Joseph Evans Assignee: Robert Joseph Evans Some blobs (like jar files) really should be tied to the life cycle of a worker. If a new blob is ready the worker should be restarted. Otherwise there is no way to pick up the contents of the newly downloaded blob. STORM-2438 already sets the ground work for this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (STORM-2438) on-demand resource requirement scaling
[ https://issues.apache.org/jira/browse/STORM-2438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated STORM-2438: -- Labels: pull-request-available (was: ) > on-demand resource requirement scaling > -- > > Key: STORM-2438 > URL: https://issues.apache.org/jira/browse/STORM-2438 > Project: Apache Storm > Issue Type: Improvement > Components: storm-core >Affects Versions: 2.0.0 >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > Labels: pull-request-available > > As a first step towards true elasticity in a storm topology we propose > allowing rebalance to also modify the resource requirements for each > bolt/spout in the topology. It will not be automatic, but it will let users > scale up and down the CPU/memory needed for a component. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (STORM-2718) Show some descriptions on LogViewer index page
[ https://issues.apache.org/jira/browse/STORM-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Li reassigned STORM-2718: --- Assignee: Ethan Li > Show some descriptions on LogViewer index page > -- > > Key: STORM-2718 > URL: https://issues.apache.org/jira/browse/STORM-2718 > Project: Apache Storm > Issue Type: Improvement >Reporter: Ethan Li >Assignee: Ethan Li >Priority: Minor > > I remembered when I was pretty new to storm and I launched logviewer for the > first time. I visited x:8000 out of my natural but then got "page not > found". I thought there must be something wrong with my storm setup. It took > me a while to figure it out that logviewer doesn't work that way. I also > found some other people were also confused about this, > [example|https://stackoverflow.com/questions/25538327/storm-logviewer-page-not-found] > . We probably want to show some descriptions/guides on the index page. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (STORM-2758) logviewer_search page not found
[ https://issues.apache.org/jira/browse/STORM-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Li updated STORM-2758: Description: I was able to search logs using search/deepSearch. For example, [^Screen Shot 2017-09-26 at 11.35.17 AM.png] But the link doesn't work. I got 404 not found when I clicked on the result link. [^Screen Shot 2017-09-26 at 11.35.03 AM.png] was: I was able to search logs using search/deepSearch. For example, [^Screen Shot 2017-09-26 at 11.35.17 AM.png] But the link doesn't work. I got 404 not found when I clicked the result. [^Screen Shot 2017-09-26 at 11.35.03 AM.png] > logviewer_search page not found > --- > > Key: STORM-2758 > URL: https://issues.apache.org/jira/browse/STORM-2758 > Project: Apache Storm > Issue Type: Bug >Reporter: Ethan Li >Assignee: Ethan Li >Priority: Minor > Labels: pull-request-available > Attachments: Screen Shot 2017-09-26 at 11.35.03 AM.png, Screen Shot > 2017-09-26 at 11.35.17 AM.png > > Time Spent: 10m > Remaining Estimate: 0h > > I was able to search logs using search/deepSearch. > For example, > [^Screen Shot 2017-09-26 at 11.35.17 AM.png] > But the link doesn't work. I got 404 not found when I clicked on the result > link. > [^Screen Shot 2017-09-26 at 11.35.03 AM.png] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (STORM-2758) logviewer_search page not found
[ https://issues.apache.org/jira/browse/STORM-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated STORM-2758: -- Labels: pull-request-available (was: ) > logviewer_search page not found > --- > > Key: STORM-2758 > URL: https://issues.apache.org/jira/browse/STORM-2758 > Project: Apache Storm > Issue Type: Bug >Reporter: Ethan Li >Assignee: Ethan Li >Priority: Minor > Labels: pull-request-available > Attachments: Screen Shot 2017-09-26 at 11.35.03 AM.png, Screen Shot > 2017-09-26 at 11.35.17 AM.png > > > I was able to search logs using search/deepSearch. > For example, > [^Screen Shot 2017-09-26 at 11.35.17 AM.png] > But the link doesn't work. I got 404 not found when I clicked the result. > [^Screen Shot 2017-09-26 at 11.35.03 AM.png] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (STORM-2758) logviewer_search page not found
[ https://issues.apache.org/jira/browse/STORM-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Li updated STORM-2758: Description: I was able to search logs using search/deepSearch. For example, [^Screen Shot 2017-09-26 at 11.35.17 AM.png] But the link doesn't work. I got 404 not found when I clicked the result. [^Screen Shot 2017-09-26 at 11.35.03 AM.png] was: I was able to search logs using search/deepSearch. For example, But the link doesn't work. I got 404 not found when I clicked the result. > logviewer_search page not found > --- > > Key: STORM-2758 > URL: https://issues.apache.org/jira/browse/STORM-2758 > Project: Apache Storm > Issue Type: Bug >Reporter: Ethan Li >Assignee: Ethan Li >Priority: Minor > Attachments: Screen Shot 2017-09-26 at 11.35.03 AM.png, Screen Shot > 2017-09-26 at 11.35.17 AM.png > > > I was able to search logs using search/deepSearch. > For example, > [^Screen Shot 2017-09-26 at 11.35.17 AM.png] > But the link doesn't work. I got 404 not found when I clicked the result. > [^Screen Shot 2017-09-26 at 11.35.03 AM.png] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (STORM-2758) logviewer_search page not found
Ethan Li created STORM-2758: --- Summary: logviewer_search page not found Key: STORM-2758 URL: https://issues.apache.org/jira/browse/STORM-2758 Project: Apache Storm Issue Type: Bug Reporter: Ethan Li Assignee: Ethan Li Priority: Minor Attachments: Screen Shot 2017-09-26 at 11.35.03 AM.png, Screen Shot 2017-09-26 at 11.35.17 AM.png I was able to search logs using search/deepSearch. For example, But the link doesn't work. I got 404 not found when I clicked the result. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
