date:20150819


 [ 
https://issues.apache.org/jira/browse/STORM-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yvonne Ironberg updated STORM-1000:
---
Description: 
In Java, the difference between “static member class” and “nonstatic member 
class” is simply a reference to  its enclosing instance. “Static” here means 
“independent of the enclosing instance”. Or “an enclosed instance can survive 
without an enclosing instance”.
* For an instance of a nonstatic member class, there is a reference from the 
enclosed instance to the enclosing instance. When the enclosing instance 
doesn’t exist, you cannot instantiate the nonstatic member class.
* For an instance of a static member class, there isn’t such a reference from 
the enclosed instance to the enclosing instance. So this helps the enclosing 
instance be garbage-collected.

Favoring static member classes when permitted improves performance because time 
and space for extra references are saved.

This optimization was done before for Storm (e.g., as part of STORM-797 
(https://issues.apache.org/jira/browse/STORM-797), see its patch 
(https://patch-diff.githubusercontent.com/raw/apache/storm/pull/537.patch).

This issue tries to improve all such places in the current codebase. 

  was:
In Java, the difference between “static member class” and “nonstatic member 
class” is simply a reference to enclosing instance. “Static” here means 
“independent of the enclosing instance”. Or “an enclosed instance can survive 
without an enclosing instance”.
* For an instance of a nonstatic member class, there is a reference from the 
enclosed instance to the enclosing instance. When the enclosing instance 
doesn’t exist, you cannot instantiate the nonstatic member class.
* For an instance of a static member class, there isn’t such a reference from 
the enclosed instance to the enclosing instance. So this helps the enclosing 
instance be garbage-collected.

Favoring static member classes when permitted improves performance because time 
and space for extra references are saved.

This optimization was done before for Storm (e.g., as part of STORM-797 
(https://issues.apache.org/jira/browse/STORM-797), see its patch 
(https://patch-diff.githubusercontent.com/raw/apache/storm/pull/537.patch).

This issue tries to improve all such places in the current codebase. 


 Use static member classes when permitted
 

 Key: STORM-1000
 URL: https://issues.apache.org/jira/browse/STORM-1000
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Yvonne Ironberg
Assignee: Yvonne Ironberg
Priority: Minor
 Attachments: STORM-1000.patch


 In Java, the difference between “static member class” and “nonstatic member 
 class” is simply a reference to  its enclosing instance. “Static” here means 
 “independent of the enclosing instance”. Or “an enclosed instance can survive 
 without an enclosing instance”.
 * For an instance of a nonstatic member class, there is a reference from the 
 enclosed instance to the enclosing instance. When the enclosing instance 
 doesn’t exist, you cannot instantiate the nonstatic member class.
 * For an instance of a static member class, there isn’t such a reference from 
 the enclosed instance to the enclosing instance. So this helps the enclosing 
 instance be garbage-collected.
 Favoring static member classes when permitted improves performance because 
 time and space for extra references are saved.
 This optimization was done before for Storm (e.g., as part of STORM-797 
 (https://issues.apache.org/jira/browse/STORM-797), see its patch 
 (https://patch-diff.githubusercontent.com/raw/apache/storm/pull/537.patch).
 This issue tries to improve all such places in the current codebase. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-919) Gathering worker and supervisor process information (CPU/Memory) and ranking the components


 [ 
https://issues.apache.org/jira/browse/STORM-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shyam Rajendran updated STORM-919:
--
Attachment: WorkerStats With Component Rank.png

 Gathering worker and supervisor process information (CPU/Memory) and ranking 
 the components
 ---

 Key: STORM-919
 URL: https://issues.apache.org/jira/browse/STORM-919
 Project: Apache Storm
  Issue Type: New Feature
Reporter: Shyam Rajendran
Assignee: Shyam Rajendran
Priority: Minor
 Attachments: WorkerStats With Component Rank.png


 It would be useful to have supervisor and worker process related information 
 such as %cpu utilization, JVM memory and network bandwidth available to 
 NIMBUS which would be useful for resource aware scheduler implementation 
 later on. As a beginning, the information can be piggybacked on the existing 
 heartbeats into the ZK or to the pacemaker as required. 
 Related JIRAs
 STORM-177
 STORM-891
 STORM-899



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-919) Gathering worker and supervisor process information (CPU/Memory)


[ 
https://issues.apache.org/jira/browse/STORM-919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703471#comment-14703471
 ] 

Shyam Rajendran commented on STORM-919:
---

*** DO NOT MERGE THIS YET  
As part of the work for Elasticity in Storm, the initial work to gather the 
stats and understand the relative impact of components has been prototyped. 
Presently only the CPU Utilization% is used to rank the components as per the 
details in previous comment.
Also the present code depends on features developed internally which are not 
yet available in the community storm. We have plans to have it merged back and 
I would like to wait till then before I can open a new request and void this.  
It would be helpful to hear opinion / suggestions on this. Thanks!

 Gathering worker and supervisor process information (CPU/Memory)
 

 Key: STORM-919
 URL: https://issues.apache.org/jira/browse/STORM-919
 Project: Apache Storm
  Issue Type: New Feature
Reporter: Shyam Rajendran
Assignee: Shyam Rajendran
Priority: Minor
 Attachments: WorkerStats With Component Rank.png


 It would be useful to have supervisor and worker process related information 
 such as %cpu utilization, JVM memory and network bandwidth available to 
 NIMBUS which would be useful for resource aware scheduler implementation 
 later on. As a beginning, the information can be piggybacked on the existing 
 heartbeats into the ZK or to the pacemaker as required. 
 Related JIRAs
 STORM-177
 STORM-891
 STORM-899



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1000) Use static member classes when permitted

2015-08-19 Thread Derek Dagit (JIRA)


[ 
https://issues.apache.org/jira/browse/STORM-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703431#comment-14703431
 ] 

Derek Dagit commented on STORM-1000:


Yeah, just create a different git branch for each JIRA Issue.  Then you can 
make a pull request for each.

 Use static member classes when permitted
 

 Key: STORM-1000
 URL: https://issues.apache.org/jira/browse/STORM-1000
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Yvonne Ironberg
Assignee: Yvonne Ironberg
Priority: Minor
 Attachments: STORM-1000.patch


 In Java, the difference between “static member class” and “nonstatic member 
 class” is simply a reference to  its enclosing instance. “Static” here means 
 “independent of the enclosing instance”. Or “an enclosed instance can survive 
 without an enclosing instance”.
 * For an instance of a nonstatic member class, there is a reference from the 
 enclosed instance to the enclosing instance. When the enclosing instance 
 doesn’t exist, you cannot instantiate the nonstatic member class.
 * For an instance of a static member class, there isn’t such a reference from 
 the enclosed instance to the enclosing instance. So this helps the enclosing 
 instance be garbage-collected.
 Favoring static member classes when permitted improves performance because 
 time and space for extra references are saved.
 This optimization was done before for Storm (e.g., as part of STORM-797), see 
 its patch 
 (https://patch-diff.githubusercontent.com/raw/apache/storm/pull/537.patch).
 This issue tries to improve all such places in the current codebase. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1000) Use static member classes when permitted


[ 
https://issues.apache.org/jira/browse/STORM-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703379#comment-14703379
 ] 

Yvonne Ironberg commented on STORM-1000:


Hi Derek, Thanks. Is it possible to have multiple pulls at the same time? I 
searched it for a while but couldn't find a way.

Now I have another pull awaiting review 
(https://github.com/apache/storm/pull/689).

 Use static member classes when permitted
 

 Key: STORM-1000
 URL: https://issues.apache.org/jira/browse/STORM-1000
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Yvonne Ironberg
Assignee: Yvonne Ironberg
Priority: Minor
 Attachments: STORM-1000.patch


 In Java, the difference between “static member class” and “nonstatic member 
 class” is simply a reference to enclosing instance. “Static” here means 
 “independent of the enclosing instance”. Or “an enclosed instance can survive 
 without an enclosing instance”.
 * For an instance of a nonstatic member class, there is a reference from the 
 enclosed instance to the enclosing instance. When the enclosing instance 
 doesn’t exist, you cannot instantiate the nonstatic member class.
 * For an instance of a static member class, there isn’t such a reference from 
 the enclosed instance to the enclosing instance. So this helps the enclosing 
 instance be garbage-collected.
 Favoring static member classes when permitted improves performance because 
 time and space for extra references are saved.
 This optimization was done before for Storm (e.g., as part of STORM-797 
 (https://issues.apache.org/jira/browse/STORM-797), see its patch 
 (https://patch-diff.githubusercontent.com/raw/apache/storm/pull/537.patch).
 This issue tries to improve all such places in the current codebase. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (STORM-919) Gathering worker and supervisor process information (CPU/Memory)

[
https://issues.apache.org/jira/browse/STORM-919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703471#comment-14703471
]

Shyam Rajendran edited comment on STORM-919 at 8/19/15 5:59 PM:

*** DO NOT MERGE THIS YET
As part of the work for Elasticity in Storm, the initial work to gather the
stats and understand the relative impact of components has been prototyped.
Presently only the CPU Utilization% is used to rank the components scheduled
across the nodes on different workers as per the details in previous comment.
Also the present code depends on features developed internally which are not
yet available in the community storm. We have plans to have it merged back and
I would like to wait till then before I can open a new request and void this.
It would be helpful to hear opinion / suggestions on this. Thanks!

was (Author: shyamrajendran):
*** DO NOT MERGE THIS YET
As part of the work for Elasticity in Storm, the initial work to gather the
stats and understand the relative impact of components has been prototyped.
Presently only the CPU Utilization% is used to rank the components as per the
details in previous comment.
Also the present code depends on features developed internally which are not
yet available in the community storm. We have plans to have it merged back and
I would like to wait till then before I can open a new request and void this.
It would be helpful to hear opinion / suggestions on this. Thanks!

Gathering worker and supervisor process information (CPU/Memory)

Key: STORM-919
URL: https://issues.apache.org/jira/browse/STORM-919
Project: Apache Storm
Issue Type: New Feature
Reporter: Shyam Rajendran
Assignee: Shyam Rajendran
Priority: Minor
Attachments: WorkerStats With Component Rank.png

It would be useful to have supervisor and worker process related information
such as %cpu utilization, JVM memory and network bandwidth available to
NIMBUS which would be useful for resource aware scheduler implementation
later on. As a beginning, the information can be piggybacked on the existing
heartbeats into the ZK or to the pacemaker as required.
Related JIRAs
STORM-177
STORM-891
STORM-899

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-919 Gathering worker and supervisor proc...

2015-08-19 Thread bourneagain

Github user bourneagain commented on the pull request:

https://github.com/apache/storm/pull/608#issuecomment-132722067
  
*** PLEASE DO NOT MERGE THIS YET  
More information @ https://issues.apache.org/jira/browse/STORM-919. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-919) Gathering worker and supervisor process information (CPU/Memory)


[ 
https://issues.apache.org/jira/browse/STORM-919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703475#comment-14703475
 ] 

ASF GitHub Bot commented on STORM-919:
--

Github user bourneagain commented on the pull request:

https://github.com/apache/storm/pull/608#issuecomment-132722067
  
*** PLEASE DO NOT MERGE THIS YET  
More information @ https://issues.apache.org/jira/browse/STORM-919. 


 Gathering worker and supervisor process information (CPU/Memory)
 

 Key: STORM-919
 URL: https://issues.apache.org/jira/browse/STORM-919
 Project: Apache Storm
  Issue Type: New Feature
Reporter: Shyam Rajendran
Assignee: Shyam Rajendran
Priority: Minor
 Attachments: WorkerStats With Component Rank.png


 It would be useful to have supervisor and worker process related information 
 such as %cpu utilization, JVM memory and network bandwidth available to 
 NIMBUS which would be useful for resource aware scheduler implementation 
 later on. As a beginning, the information can be piggybacked on the existing 
 heartbeats into the ZK or to the pacemaker as required. 
 Related JIRAs
 STORM-177
 STORM-891
 STORM-899



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-1000) Use static member classes when permitted


 [ 
https://issues.apache.org/jira/browse/STORM-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yvonne Ironberg updated STORM-1000:
---
Description: 
In Java, the difference between “static member class” and “nonstatic member 
class” is simply a reference to  its enclosing instance. “Static” here means 
“independent of the enclosing instance”. Or “an enclosed instance can survive 
without an enclosing instance”.
* For an instance of a nonstatic member class, there is a reference from the 
enclosed instance to the enclosing instance. When the enclosing instance 
doesn’t exist, you cannot instantiate the nonstatic member class.
* For an instance of a static member class, there isn’t such a reference from 
the enclosed instance to the enclosing instance. So this helps the enclosing 
instance be garbage-collected.

Favoring static member classes when permitted improves performance because time 
and space for extra references are saved.

This optimization was done before for Storm (e.g., as part of STORM-797), see 
its patch 
(https://patch-diff.githubusercontent.com/raw/apache/storm/pull/537.patch).

This issue tries to improve all such places in the current codebase. 

  was:
In Java, the difference between “static member class” and “nonstatic member 
class” is simply a reference to  its enclosing instance. “Static” here means 
“independent of the enclosing instance”. Or “an enclosed instance can survive 
without an enclosing instance”.
* For an instance of a nonstatic member class, there is a reference from the 
enclosed instance to the enclosing instance. When the enclosing instance 
doesn’t exist, you cannot instantiate the nonstatic member class.
* For an instance of a static member class, there isn’t such a reference from 
the enclosed instance to the enclosing instance. So this helps the enclosing 
instance be garbage-collected.

Favoring static member classes when permitted improves performance because time 
and space for extra references are saved.

This optimization was done before for Storm (e.g., as part of STORM-797 
(https://issues.apache.org/jira/browse/STORM-797), see its patch 
(https://patch-diff.githubusercontent.com/raw/apache/storm/pull/537.patch).

This issue tries to improve all such places in the current codebase. 


 Use static member classes when permitted
 

 Key: STORM-1000
 URL: https://issues.apache.org/jira/browse/STORM-1000
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Yvonne Ironberg
Assignee: Yvonne Ironberg
Priority: Minor
 Attachments: STORM-1000.patch


 In Java, the difference between “static member class” and “nonstatic member 
 class” is simply a reference to  its enclosing instance. “Static” here means 
 “independent of the enclosing instance”. Or “an enclosed instance can survive 
 without an enclosing instance”.
 * For an instance of a nonstatic member class, there is a reference from the 
 enclosed instance to the enclosing instance. When the enclosing instance 
 doesn’t exist, you cannot instantiate the nonstatic member class.
 * For an instance of a static member class, there isn’t such a reference from 
 the enclosed instance to the enclosing instance. So this helps the enclosing 
 instance be garbage-collected.
 Favoring static member classes when permitted improves performance because 
 time and space for extra references are saved.
 This optimization was done before for Storm (e.g., as part of STORM-797), see 
 its patch 
 (https://patch-diff.githubusercontent.com/raw/apache/storm/pull/537.patch).
 This issue tries to improve all such places in the current codebase. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-919) Gathering worker and supervisor process information (CPU/Memory)


 [ 
https://issues.apache.org/jira/browse/STORM-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shyam Rajendran updated STORM-919:
--
Summary: Gathering worker and supervisor process information (CPU/Memory)  
(was: Gathering worker and supervisor process information (CPU/Memory) and rank 
the components)

 Gathering worker and supervisor process information (CPU/Memory)
 

 Key: STORM-919
 URL: https://issues.apache.org/jira/browse/STORM-919
 Project: Apache Storm
  Issue Type: New Feature
Reporter: Shyam Rajendran
Assignee: Shyam Rajendran
Priority: Minor
 Attachments: WorkerStats With Component Rank.png


 It would be useful to have supervisor and worker process related information 
 such as %cpu utilization, JVM memory and network bandwidth available to 
 NIMBUS which would be useful for resource aware scheduler implementation 
 later on. As a beginning, the information can be piggybacked on the existing 
 heartbeats into the ZK or to the pacemaker as required. 
 Related JIRAs
 STORM-177
 STORM-891
 STORM-899



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-893) Resource Aware Scheduling

2015-08-19 Thread Boyang Jerry Peng (JIRA)

[
https://issues.apache.org/jira/browse/STORM-893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703632#comment-14703632
]

Boyang Jerry Peng commented on STORM-893:
-

Hello Everyone,

I recently published a paper in ACM Middleware on the subject of Resource Aware
Scheduling in Storm in which I demonstrated how the scheduling algorithms I
used can significantly improve performance and stability of Storm. I have
created an implementation for open source based off of a implementation I
worked on a Yahoo. The implementation is on my github. The link is provided
below:

https://github.com/jerrypeng/storm/tree/opensource_ras

The link to the paper:

http://web.engr.illinois.edu/~bpeng/files/r-storm.pdf

The paper describes the architecture, algorithms, and API of the Resource Aware
Scheduler as well as an evaluation of performance. The research and initial
implementation was done by me while I was a graduate student at University of
Illinois, Urbana-champaign. After I graduated, I joined Yahoo and we came up
with a better implementation.

The implementation should work straight out of the box. If you are running
storm in a multi-rack environment, and you want to leverage the network aware
portion of the scheduler to improve performance and decrease latency, you can
implement your own STORM_NETWORK_TOPOLOGY_PLUGIN that maps the underlying
physical network. The default plugin that comes with the implementation
assumes all machines have the same network distance from each other i.e. on the
same rack. Contact me if you need more information on this subject.

Please look at this initial implementation and provide me with some feedback,
since we intend to have a pull request up soon and merge it into master.

People that worked on the implementation at Yahoo with me:

Bobby Evans (Yahoo Storm PMC)
Derek Dagit (Yahoo Storm PMC)
Kyle Nusbaum (Yahoo Storm PMC)
Liu Zhuo (Yahoo)
Sanket Chintapalli (Yahoo)
Reza Fravier (Yahoo UIUC)

Best,

Boyang Jerry Peng
Software Engineer, Yahoo
http://web.engr.illinois.edu/~bpeng/

Resource Aware Scheduling
-

Key: STORM-893
URL: https://issues.apache.org/jira/browse/STORM-893
Project: Apache Storm
Issue Type: Umbrella
Reporter: Robert Joseph Evans
Assignee: Boyang Jerry Peng

At Yahoo we have been working on resource aware scheduling in storm, based
off of some work done in academia. This rollup ticket is to track the
complete project. With several sub tasks. Some that are already done and
need to be pushed back, and others that we have not started on yet.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

R-Storm - Resource Aware Scheduling implementation for Storm

2015-08-19 Thread Boyang(Jerry) Peng

Hello Everyone,



I recently published a paper in ACM Middleware on the subject of Resource Aware 
Scheduling in Storm in which I demonstrated how the scheduling algorithms I 
used can significantly improve performance and stability of Storm.  I have 
created an implementation for open source based off of a implementation I 
worked on a Yahoo. The implementation is on my github. The link is provided 
below:
https://github.com/jerrypeng/storm/tree/opensource_ras

The link to the paper: 
http://web.engr.illinois.edu/~bpeng/files/r-storm.pdf

The paper describes the architecture, algorithms, and API of the Resource Aware 
Scheduler as well as an evaluation of performance.  The research and initial 
implementation was done by me while I was a graduate student at University of 
Illinois, Urbana-champaign.  After I graduated, I joined Yahoo and we came up 
with a better implementation.
The implementation should work straight out of the box.  If you are running 
storm in a multi-rack environment, and you want to leverage the network aware 
portion of the scheduler to improve performance and decrease latency, you can 
implement your own STORM_NETWORK_TOPOLOGY_PLUGIN that maps the underlying 
physical network.  The default plugin that comes with the implementation 
assumes all machines have the same network distance from each other i.e. on the 
same rack.  Contact me if you need more information on this subject.
Please look at this initial implementation and provide me with some feedback, 
since we intend to have a pull request up soon and merge it into master.
People that worked on the implementation at Yahoo with me:

Bobby Evans (Yahoo  Storm PMC)Derek Dagit (Yahoo  Storm PMC)Kyle Nusbaum 
(Yahoo  Storm PMC)
Liu Zhuo (Yahoo)Sanket Chintapalli (Yahoo)Reza Fravier (Yahoo  UIUC)
Best,
Boyang Jerry PengSoftware Engineer, Yahoohttp://web.engr.illinois.edu/~bpeng/

[GitHub] storm pull request: Storm-166: Nimbus HA design doc and implementa...

2015-08-19 Thread Parth-Brahmbhatt

Github user Parth-Brahmbhatt commented on the pull request:

https://github.com/apache/storm/pull/354#issuecomment-132737685
  
@revans2 Can you please review this PR when you have time? Given you have 
been involved in the original PR I don't want to commit this until I get a 
confirmation from you. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-996) netty-unit-tests/test-batch demonstrates out-of-order delivery

2015-08-19 Thread Parth Brahmbhatt (JIRA)


[ 
https://issues.apache.org/jira/browse/STORM-996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703522#comment-14703522
 ] 

Parth Brahmbhatt commented on STORM-996:


[~dagit] How are you reproducing this? I just ran this test in a loop for 50 
times and it passed all the time for me. I am on following java version

➜  incubator-storm git:(ha-merge) ✗ java -version
java version 1.8.0_31
Java(TM) SE Runtime Environment (build 1.8.0_31-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode)




 netty-unit-tests/test-batch demonstrates out-of-order delivery
 --

 Key: STORM-996
 URL: https://issues.apache.org/jira/browse/STORM-996
 Project: Apache Storm
  Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Derek Dagit
Assignee: Derek Dagit
Priority: Blocker

 backtype.storm.messaging.netty-unit-test/test-batch
 One example of output.  Similar things happen sporadically and vary widely by 
 number of failed assertions.
 Tuples are not just skewed, but actually seem to come in out-of-order.
 {quote}
 actual: (not (= 66040 66041))
 at: test_runner.clj:105
 expected: (= req_msg resp_msg)
 actual: (not (= 66041 66042))
 at: test_runner.clj:105
 expected: (= req_msg resp_msg)
 actual: (not (= 66042 66040))
 at: test_runner.clj:105
 {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1000

2015-08-19 Thread YvonneIronberg

GitHub user YvonneIronberg opened a pull request:

https://github.com/apache/storm/pull/691

STORM-1000



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/YvonneIronberg/storm storm-1000

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/691.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #691


commit 6a30be80a895c63d5e0c53183a9807b8e8c3a8b5
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-18T17:37:31Z

STORM-829.

commit 8d487852e51c67d943ee6ea2b778e9e68da5f062
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-18T17:44:29Z

STORM-829.

commit a59823446b89ccce461c72b655e157983be4dca2
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-19T19:35:42Z

STORM-1000.

commit a7cc363572ae324f5b46e05d7ce2a8efc641812d
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-19T22:09:42Z

STORM-1000.

commit 6ae2a9a88443342a6aab12bf5d5dd15e6405907b
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-19T22:33:47Z

storm-1000.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1000) Use static member classes when permitted


[ 
https://issues.apache.org/jira/browse/STORM-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703857#comment-14703857
 ] 

ASF GitHub Bot commented on STORM-1000:
---

GitHub user YvonneIronberg opened a pull request:

https://github.com/apache/storm/pull/691

STORM-1000



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/YvonneIronberg/storm storm-1000

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/691.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #691


commit 6a30be80a895c63d5e0c53183a9807b8e8c3a8b5
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-18T17:37:31Z

STORM-829.

commit 8d487852e51c67d943ee6ea2b778e9e68da5f062
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-18T17:44:29Z

STORM-829.

commit a59823446b89ccce461c72b655e157983be4dca2
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-19T19:35:42Z

STORM-1000.

commit a7cc363572ae324f5b46e05d7ce2a8efc641812d
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-19T22:09:42Z

STORM-1000.

commit 6ae2a9a88443342a6aab12bf5d5dd15e6405907b
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-19T22:33:47Z

storm-1000.




 Use static member classes when permitted
 

 Key: STORM-1000
 URL: https://issues.apache.org/jira/browse/STORM-1000
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Yvonne Ironberg
Assignee: Yvonne Ironberg
Priority: Minor
 Attachments: STORM-1000.patch


 In Java, the difference between “static member class” and “nonstatic member 
 class” is simply a reference to  its enclosing instance. “Static” here means 
 “independent of the enclosing instance”. Or “an enclosed instance can survive 
 without an enclosing instance”.
 * For an instance of a nonstatic member class, there is a reference from the 
 enclosed instance to the enclosing instance. When the enclosing instance 
 doesn’t exist, you cannot instantiate the nonstatic member class.
 * For an instance of a static member class, there isn’t such a reference from 
 the enclosed instance to the enclosing instance. So this helps the enclosing 
 instance be garbage-collected.
 Favoring static member classes when permitted improves performance because 
 time and space for extra references are saved.
 This optimization was done before for Storm (e.g., as part of STORM-797), see 
 its patch 
 (https://patch-diff.githubusercontent.com/raw/apache/storm/pull/537.patch).
 This issue tries to improve all such places in the current codebase. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-829) Hadoop dependency confusion


[ 
https://issues.apache.org/jira/browse/STORM-829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703856#comment-14703856
 ] 

ASF GitHub Bot commented on STORM-829:
--

GitHub user YvonneIronberg opened a pull request:

https://github.com/apache/storm/pull/690

STORM-829



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/YvonneIronberg/storm storm-829

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/690.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #690


commit 6a30be80a895c63d5e0c53183a9807b8e8c3a8b5
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-18T17:37:31Z

STORM-829.

commit 8d487852e51c67d943ee6ea2b778e9e68da5f062
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-18T17:44:29Z

STORM-829.

commit a59823446b89ccce461c72b655e157983be4dca2
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-19T19:35:42Z

STORM-1000.

commit 2c8502cc84565d138c97709737dbd60cdb7b2a1a
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-19T21:00:33Z

STORM-829.

commit f3deb85b8e58069cca26ec87f71f4044e86d50cf
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-19T22:23:24Z

STORM-829.

commit 0c8582e453caa12093f47c4b66c084dd28f77cd5
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-19T22:30:02Z

storm-829.

commit 095c796f6e26e555dfc3f62146721553d3d5cb35
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-19T22:32:25Z

storm-829.




 Hadoop dependency confusion
 ---

 Key: STORM-829
 URL: https://issues.apache.org/jira/browse/STORM-829
 Project: Apache Storm
  Issue Type: Bug
Reporter: Robert Joseph Evans

 The hadoop dependencies seem a bit messed up.  We have a hadoop.version set 
 in the main pom.xml to 2.6.0 but it does not really appear to be used.  
 storm-core uses hadoop-auth set to 2.4.0, storm-hdfs hard codes it to 2.2.0, 
 storm-hbase does the same.  We should have all of the versions be consistent 
 and come from the main pom.xml.  Also what are we using hadoop-auth for in 
 storm-core.
 Also it looks like we should remove the hadoop-auth dependency.  When 
 AutoHDFS and AutoHBase moved out of storm-core it looks like it was left 
 there by mistake. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1000) Use static member classes when permitted


[ 
https://issues.apache.org/jira/browse/STORM-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703861#comment-14703861
 ] 

Yvonne Ironberg commented on STORM-1000:


Thanks, [~dagit].

Please refer to the clean version: 
https://github.com/apache/storm/pull/691/files. The patch is solely devoted for 
STORM-1000, not STORM-829. The above 
https://github.com/apache/storm/pull/691.patch is inaccurate.

 Use static member classes when permitted
 

 Key: STORM-1000
 URL: https://issues.apache.org/jira/browse/STORM-1000
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Yvonne Ironberg
Assignee: Yvonne Ironberg
Priority: Minor
 Attachments: STORM-1000.patch


 In Java, the difference between “static member class” and “nonstatic member 
 class” is simply a reference to  its enclosing instance. “Static” here means 
 “independent of the enclosing instance”. Or “an enclosed instance can survive 
 without an enclosing instance”.
 * For an instance of a nonstatic member class, there is a reference from the 
 enclosed instance to the enclosing instance. When the enclosing instance 
 doesn’t exist, you cannot instantiate the nonstatic member class.
 * For an instance of a static member class, there isn’t such a reference from 
 the enclosed instance to the enclosing instance. So this helps the enclosing 
 instance be garbage-collected.
 Favoring static member classes when permitted improves performance because 
 time and space for extra references are saved.
 This optimization was done before for Storm (e.g., as part of STORM-797), see 
 its patch 
 (https://patch-diff.githubusercontent.com/raw/apache/storm/pull/537.patch).
 This issue tries to improve all such places in the current codebase. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-829) Hadoop dependency confusion


[ 
https://issues.apache.org/jira/browse/STORM-829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703862#comment-14703862
 ] 

Yvonne Ironberg commented on STORM-829:
---

The only changes are in https://github.com/apache/storm/pull/690/files.

 Hadoop dependency confusion
 ---

 Key: STORM-829
 URL: https://issues.apache.org/jira/browse/STORM-829
 Project: Apache Storm
  Issue Type: Bug
Reporter: Robert Joseph Evans

 The hadoop dependencies seem a bit messed up.  We have a hadoop.version set 
 in the main pom.xml to 2.6.0 but it does not really appear to be used.  
 storm-core uses hadoop-auth set to 2.4.0, storm-hdfs hard codes it to 2.2.0, 
 storm-hbase does the same.  We should have all of the versions be consistent 
 and come from the main pom.xml.  Also what are we using hadoop-auth for in 
 storm-core.
 Also it looks like we should remove the hadoop-auth dependency.  When 
 AutoHDFS and AutoHBase moved out of storm-core it looks like it was left 
 there by mistake. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-829

2015-08-19 Thread YvonneIronberg

GitHub user YvonneIronberg opened a pull request:

https://github.com/apache/storm/pull/690

STORM-829



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/YvonneIronberg/storm storm-829

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/690.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #690


commit 6a30be80a895c63d5e0c53183a9807b8e8c3a8b5
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-18T17:37:31Z

STORM-829.

commit 8d487852e51c67d943ee6ea2b778e9e68da5f062
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-18T17:44:29Z

STORM-829.

commit a59823446b89ccce461c72b655e157983be4dca2
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-19T19:35:42Z

STORM-1000.

commit 2c8502cc84565d138c97709737dbd60cdb7b2a1a
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-19T21:00:33Z

STORM-829.

commit f3deb85b8e58069cca26ec87f71f4044e86d50cf
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-19T22:23:24Z

STORM-829.

commit 0c8582e453caa12093f47c4b66c084dd28f77cd5
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-19T22:30:02Z

storm-829.

commit 095c796f6e26e555dfc3f62146721553d3d5cb35
Author: YvonneIronberg yvonne.ironb...@gmail.com
Date:   2015-08-19T22:32:25Z

storm-829.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [STORM-949] On the topology summary UI page, a...

2015-08-19 Thread jerrypeng

Github user jerrypeng commented on the pull request:

https://github.com/apache/storm/pull/675#issuecomment-132891659
  
Just modified the UI to have error time shown as a time and date.  If you 
hover over that time a tooltip will pop up displaying the elapsed time.

![new_ui](https://cloud.githubusercontent.com/assets/3613359/9376134/9574276e-46ce-11e5-92e8-e3a4ef9b44c9.png)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-949) On the topology summary UI page, last shown error should have the time and date


[ 
https://issues.apache.org/jira/browse/STORM-949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704294#comment-14704294
 ] 

ASF GitHub Bot commented on STORM-949:
--

Github user jerrypeng commented on the pull request:

https://github.com/apache/storm/pull/675#issuecomment-132891659
  
Just modified the UI to have error time shown as a time and date.  If you 
hover over that time a tooltip will pop up displaying the elapsed time.

![new_ui](https://cloud.githubusercontent.com/assets/3613359/9376134/9574276e-46ce-11e5-92e8-e3a4ef9b44c9.png)



 On the topology summary UI page, last shown error should have the time and 
 date
 ---

 Key: STORM-949
 URL: https://issues.apache.org/jira/browse/STORM-949
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Reza Farivar
Assignee: Boyang Jerry Peng
Priority: Minor

 Without showing the time and date, it is not clear whether the error is a 
 recent one or an old stale one. 
 (Error text is colored red when it is in last 5 minutes, but this is not 
 obvious and not enough information)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (STORM-1000) Use static member classes when permitted

Yvonne Ironberg created STORM-1000:
--

 Summary: Use static member classes when permitted
 Key: STORM-1000
 URL: https://issues.apache.org/jira/browse/STORM-1000
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Yvonne Ironberg
Assignee: Yvonne Ironberg
Priority: Minor


In Java, the difference between “static member class” and “nonstatic member 
class” is simply a reference to enclosing instance. “Static” here means 
“independent of the enclosing instance”. Or “an enclosed instance can survive 
without an enclosing instance”.
* For an instance of a nonstatic member class, there is a reference from the 
enclosed instance to the enclosing instance. When the enclosing instance 
doesn’t exist, you cannot instantiate the nonstatic member class.
* For an instance of a static member class, there isn’t such a reference from 
the enclosed instance to the enclosing instance. So this helps the enclosing 
instance be garbage-collected.

Favoring static member classes when permitted improves performance because time 
and space for extra references are saved.

This optimization was done before for Storm (e.g., as part of 
(https://issues.apache.org/jira/browse/STORM-797)[STORM-797], see 
(https://patch-diff.githubusercontent.com/raw/apache/storm/pull/537.patch)[its 
patch]).

This issue tries to improve all such places in the current codebase. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-1000) Use static member classes when permitted


 [ 
https://issues.apache.org/jira/browse/STORM-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yvonne Ironberg updated STORM-1000:
---
Description: 
In Java, the difference between “static member class” and “nonstatic member 
class” is simply a reference to enclosing instance. “Static” here means 
“independent of the enclosing instance”. Or “an enclosed instance can survive 
without an enclosing instance”.
* For an instance of a nonstatic member class, there is a reference from the 
enclosed instance to the enclosing instance. When the enclosing instance 
doesn’t exist, you cannot instantiate the nonstatic member class.
* For an instance of a static member class, there isn’t such a reference from 
the enclosed instance to the enclosing instance. So this helps the enclosing 
instance be garbage-collected.

Favoring static member classes when permitted improves performance because time 
and space for extra references are saved.

This optimization was done before for Storm (e.g., as part of STORM-797 
(https://issues.apache.org/jira/browse/STORM-797), see its patch 
(https://patch-diff.githubusercontent.com/raw/apache/storm/pull/537.patch).

This issue tries to improve all such places in the current codebase. 

  was:
In Java, the difference between “static member class” and “nonstatic member 
class” is simply a reference to enclosing instance. “Static” here means 
“independent of the enclosing instance”. Or “an enclosed instance can survive 
without an enclosing instance”.
* For an instance of a nonstatic member class, there is a reference from the 
enclosed instance to the enclosing instance. When the enclosing instance 
doesn’t exist, you cannot instantiate the nonstatic member class.
* For an instance of a static member class, there isn’t such a reference from 
the enclosed instance to the enclosing instance. So this helps the enclosing 
instance be garbage-collected.

Favoring static member classes when permitted improves performance because time 
and space for extra references are saved.

This optimization was done before for Storm (e.g., as part of 
(https://issues.apache.org/jira/browse/STORM-797)[STORM-797], see 
(https://patch-diff.githubusercontent.com/raw/apache/storm/pull/537.patch)[its 
patch]).

This issue tries to improve all such places in the current codebase. 


 Use static member classes when permitted
 

 Key: STORM-1000
 URL: https://issues.apache.org/jira/browse/STORM-1000
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Yvonne Ironberg
Assignee: Yvonne Ironberg
Priority: Minor

 In Java, the difference between “static member class” and “nonstatic member 
 class” is simply a reference to enclosing instance. “Static” here means 
 “independent of the enclosing instance”. Or “an enclosed instance can survive 
 without an enclosing instance”.
 * For an instance of a nonstatic member class, there is a reference from the 
 enclosed instance to the enclosing instance. When the enclosing instance 
 doesn’t exist, you cannot instantiate the nonstatic member class.
 * For an instance of a static member class, there isn’t such a reference from 
 the enclosed instance to the enclosing instance. So this helps the enclosing 
 instance be garbage-collected.
 Favoring static member classes when permitted improves performance because 
 time and space for extra references are saved.
 This optimization was done before for Storm (e.g., as part of STORM-797 
 (https://issues.apache.org/jira/browse/STORM-797), see its patch 
 (https://patch-diff.githubusercontent.com/raw/apache/storm/pull/537.patch).
 This issue tries to improve all such places in the current codebase. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-1000) Use static member classes when permitted


 [ 
https://issues.apache.org/jira/browse/STORM-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yvonne Ironberg updated STORM-1000:
---
Attachment: STORM-1000.patch

 Use static member classes when permitted
 

 Key: STORM-1000
 URL: https://issues.apache.org/jira/browse/STORM-1000
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Yvonne Ironberg
Assignee: Yvonne Ironberg
Priority: Minor
 Attachments: STORM-1000.patch


 In Java, the difference between “static member class” and “nonstatic member 
 class” is simply a reference to enclosing instance. “Static” here means 
 “independent of the enclosing instance”. Or “an enclosed instance can survive 
 without an enclosing instance”.
 * For an instance of a nonstatic member class, there is a reference from the 
 enclosed instance to the enclosing instance. When the enclosing instance 
 doesn’t exist, you cannot instantiate the nonstatic member class.
 * For an instance of a static member class, there isn’t such a reference from 
 the enclosed instance to the enclosing instance. So this helps the enclosing 
 instance be garbage-collected.
 Favoring static member classes when permitted improves performance because 
 time and space for extra references are saved.
 This optimization was done before for Storm (e.g., as part of STORM-797 
 (https://issues.apache.org/jira/browse/STORM-797), see its patch 
 (https://patch-diff.githubusercontent.com/raw/apache/storm/pull/537.patch).
 This issue tries to improve all such places in the current codebase. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-1000) Use static member classes when permitted


 [ 
https://issues.apache.org/jira/browse/STORM-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yvonne Ironberg updated STORM-1000:
---
Attachment: (was: STORM-1000.patch)

 Use static member classes when permitted
 

 Key: STORM-1000
 URL: https://issues.apache.org/jira/browse/STORM-1000
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Yvonne Ironberg
Assignee: Yvonne Ironberg
Priority: Minor
 Attachments: STORM-1000.patch


 In Java, the difference between “static member class” and “nonstatic member 
 class” is simply a reference to enclosing instance. “Static” here means 
 “independent of the enclosing instance”. Or “an enclosed instance can survive 
 without an enclosing instance”.
 * For an instance of a nonstatic member class, there is a reference from the 
 enclosed instance to the enclosing instance. When the enclosing instance 
 doesn’t exist, you cannot instantiate the nonstatic member class.
 * For an instance of a static member class, there isn’t such a reference from 
 the enclosed instance to the enclosing instance. So this helps the enclosing 
 instance be garbage-collected.
 Favoring static member classes when permitted improves performance because 
 time and space for extra references are saved.
 This optimization was done before for Storm (e.g., as part of STORM-797 
 (https://issues.apache.org/jira/browse/STORM-797), see its patch 
 (https://patch-diff.githubusercontent.com/raw/apache/storm/pull/537.patch).
 This issue tries to improve all such places in the current codebase. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-1000) Use static member classes when permitted