[jira] [Commented] (STORM-1674) Idle KafkaSpout consumes more bandwidth than needed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15244535#comment-15244535
 ] 

ASF GitHub Bot commented on STORM-1674:
---

Github user moesol commented on the pull request:

https://github.com/apache/storm/pull/1287#issuecomment-210955517
  
@knusbaum I added the JIRA ticket and updated the title a while ago. Are 
you the person who will actually perform the merge?


> Idle KafkaSpout consumes more bandwidth than needed
> ---
>
> Key: STORM-1674
> URL: https://issues.apache.org/jira/browse/STORM-1674
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-kafka
>Affects Versions: 0.9.3
>Reporter: Robert Hastings
>
> Discovered 30 megabits of traffic flowing between a set of KafkaSpouts
> and our kafka servers even though no Kafka messages were moving.
> Using the wireshark kafka dissector, we were able to see that
> each FetchRequest had maxWait set to 1
> and minBytes set to 0. When binBytes is set to 0 the kafka server
> responds immediately when there are no messages. In turn the KafkaSpout
> polls without any delay causing a constant stream of FetchRequest/
> FetchResponse messages. Using a non-KafkaSpout client had a similar
> traffic pattern with two key differences
> 1) minBytes was 1
> 2) maxWait was 100
> With these FetchRequest parameters and no messages flowing,
> the kafka server delays the FetchResponse by 100 ms. This reduces
> the network traffic from megabits to the low kilobits. It also
> reduced the CPU utilization of our kafka server from 140% to 2%.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1674: Addresses network flood from Kafka...

[moesol]

Github user moesol commented on the pull request:

https://github.com/apache/storm/pull/1287#issuecomment-210955517
  
@knusbaum I added the JIRA ticket and updated the title a while ago. Are 
you the person who will actually perform the merge?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [STORM-1714] StatefulBolts ends up as normal b...

[arunmahadevan]

Github user arunmahadevan commented on the pull request:

https://github.com/apache/storm/pull/1343#issuecomment-210853684
  
@harshach yes both master and 1.x branch


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (STORM-1716) Add some external Jedis pool config

[darion yaphet (JIRA)]

darion yaphet created STORM-1716:


 Summary: Add some external Jedis pool config
 Key: STORM-1716
 URL: https://issues.apache.org/jira/browse/STORM-1716
 Project: Apache Storm
  Issue Type: Improvement
  Components: storm-redis
Affects Versions: 1.0.0, 0.9.6, 2.0.0
Reporter: darion yaphet
Assignee: darion yaphet
Priority: Minor


add some jedis pool config 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1704 When logviewer_search.html opens da...

[harshach]

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/1329#issuecomment-210853124
  
+1. @HeartSaVioR should we merge this for 1.x-branch as well.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1714) StatefulBolts ends up as normal bolts while using TopologyBuilder.setBolt without parallelism

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15244303#comment-15244303
 ] 

ASF GitHub Bot commented on STORM-1714:
---

Github user arunmahadevan commented on the pull request:

https://github.com/apache/storm/pull/1343#issuecomment-210853684
  
@harshach yes both master and 1.x branch


> StatefulBolts ends up as normal bolts while using TopologyBuilder.setBolt 
> without parallelism
> -
>
> Key: STORM-1714
> URL: https://issues.apache.org/jira/browse/STORM-1714
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 1.0.0, 2.0.0
>Reporter: Arun Mahadevan
>Assignee: Arun Mahadevan
>
> StatefulBolt inherits from IRichBolt which but the TopologyBuilder.setBolt 
> overload is chosen based on the static type of the parameter causing issues. 
> See if StatfulBolt can be refactored to not directly inherit from IRichBolt.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1705: Cap number of retries for a failed...

[harshach]

Github user harshach commented on a diff in the pull request:

https://github.com/apache/storm/pull/1331#discussion_r59970071
  
--- Diff: 
external/storm-kafka/src/jvm/org/apache/storm/kafka/PartitionManager.java ---
@@ -46,6 +46,9 @@
 private final CountMetric _fetchAPIMessageCount;
--- End diff --

since we are refactoring this code. Can we simplify partitionManager 
constructor instead of the current state we can have few private methods for 
all the code in the constructor.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1704) When logviewer_search.html opens daemon file, next search always show no result

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15244299#comment-15244299
 ] 

ASF GitHub Bot commented on STORM-1704:
---

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/1329#issuecomment-210853124
  
+1. @HeartSaVioR should we merge this for 1.x-branch as well.


> When logviewer_search.html opens daemon file, next search always show no 
> result
> ---
>
> Key: STORM-1704
> URL: https://issues.apache.org/jira/browse/STORM-1704
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0, 2.0.0
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
>Priority: Minor
>
> When searching keyword in /logviewer_search.html with daemon log file, 
> is-daemon=yes parameter is gone so search shows no result.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1475) storm-elasticsearch should support ES 2.X

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15244296#comment-15244296
 ] 

ASF GitHub Bot commented on STORM-1475:
---

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/1337#issuecomment-210852905
  
@dossett is there any way to merge the existing module with new one? 
instead of having another module


> storm-elasticsearch should support ES 2.X
> -
>
> Key: STORM-1475
> URL: https://issues.apache.org/jira/browse/STORM-1475
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-elasticsearch
>Affects Versions: 1.0.0
>Reporter: Aaron Dossett
>Assignee: Aaron Dossett
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1714] StatefulBolts ends up as normal b...

[harshach]

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/1343#issuecomment-210850532
  
+1. @arunmahadevan we want this in 1.x-branch as well right?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1714) StatefulBolts ends up as normal bolts while using TopologyBuilder.setBolt without parallelism

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15244292#comment-15244292
 ] 

ASF GitHub Bot commented on STORM-1714:
---

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/1343#issuecomment-210850532
  
+1. @arunmahadevan we want this in 1.x-branch as well right?


> StatefulBolts ends up as normal bolts while using TopologyBuilder.setBolt 
> without parallelism
> -
>
> Key: STORM-1714
> URL: https://issues.apache.org/jira/browse/STORM-1714
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 1.0.0, 2.0.0
>Reporter: Arun Mahadevan
>Assignee: Arun Mahadevan
>
> StatefulBolt inherits from IRichBolt which but the TopologyBuilder.setBolt 
> overload is chosen based on the static type of the parameter causing issues. 
> See if StatfulBolt can be refactored to not directly inherit from IRichBolt.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1715) Jedis Default Host

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15244270#comment-15244270
 ] 

ASF GitHub Bot commented on STORM-1715:
---

GitHub user darionyaphet opened a pull request:

https://github.com/apache/storm/pull/1345

[Storm 1715] Jedis Default Host

Useing Jedis Protocol.DEFAULT_HOST to replace DEFAULT_HOST :)

[STORM-1715](https://issues.apache.org/jira/browse/STORM-1715)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/darionyaphet/storm 1715

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1345.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1345


commit 68c0773247b36a2bba2eb0fcf4fd34130cb70862
Author: yaphet 
Date:   2016-04-01T04:00:22Z

Merge pull request #1 from apache/master

merge request

commit 8ecd79fbdde17511603137062b66a346b88165be
Author: yaphet 
Date:   2016-04-16T14:25:51Z

Merge pull request #2 from apache/master

merge request

commit b3c0e2d711962ee1629775c4f8f47fed36669633
Author: darionyaphet 
Date:   2016-04-16T15:18:02Z

using jedis's Protocol.DEFAULT_HOST replace DEFAULT_HOST




> Jedis Default Host
> --
>
> Key: STORM-1715
> URL: https://issues.apache.org/jira/browse/STORM-1715
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-redis
>Affects Versions: 1.0.0, 2.0.0
>Reporter: darion yaphet
>Assignee: darion yaphet
>Priority: Minor
> Fix For: 2.0.0
>
>
> Useing Jedis Protocol.DEFAULT_HOST to replace DEFAULT_HOST



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [Storm 1715] Jedis Default Host

[darionyaphet]

GitHub user darionyaphet opened a pull request:

https://github.com/apache/storm/pull/1345

[Storm 1715] Jedis Default Host

Useing Jedis Protocol.DEFAULT_HOST to replace DEFAULT_HOST :)

[STORM-1715](https://issues.apache.org/jira/browse/STORM-1715)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/darionyaphet/storm 1715

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1345.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1345


commit 68c0773247b36a2bba2eb0fcf4fd34130cb70862
Author: yaphet 
Date:   2016-04-01T04:00:22Z

Merge pull request #1 from apache/master

merge request

commit 8ecd79fbdde17511603137062b66a346b88165be
Author: yaphet 
Date:   2016-04-16T14:25:51Z

Merge pull request #2 from apache/master

merge request

commit b3c0e2d711962ee1629775c4f8f47fed36669633
Author: darionyaphet 
Date:   2016-04-16T15:18:02Z

using jedis's Protocol.DEFAULT_HOST replace DEFAULT_HOST




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: Storm 1715

[darionyaphet]

Github user darionyaphet closed the pull request at:

https://github.com/apache/storm/pull/1344


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: Storm 1715

[darionyaphet]

GitHub user darionyaphet opened a pull request:

https://github.com/apache/storm/pull/1344

Storm 1715



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/darionyaphet/storm STORM-1715

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1344.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1344


commit 68c0773247b36a2bba2eb0fcf4fd34130cb70862
Author: yaphet 
Date:   2016-04-01T04:00:22Z

Merge pull request #1 from apache/master

merge request

commit 8ecd79fbdde17511603137062b66a346b88165be
Author: yaphet 
Date:   2016-04-16T14:25:51Z

Merge pull request #2 from apache/master

merge request

commit 1b6b46408ca56f1fa42bb23206e78652f3dccf4b
Author: wangcg 
Date:   2016-04-16T14:37:13Z

using jedis Protocol.DEFAULT_HOST




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (STORM-1715) Jedis Default Host

[darion yaphet (JIRA)]

darion yaphet created STORM-1715:


 Summary: Jedis Default Host
 Key: STORM-1715
 URL: https://issues.apache.org/jira/browse/STORM-1715
 Project: Apache Storm
  Issue Type: Improvement
  Components: storm-redis
Affects Versions: 1.0.0, 2.0.0
Reporter: darion yaphet
Assignee: darion yaphet
Priority: Minor
 Fix For: 2.0.0


Useing Jedis Protocol.DEFAULT_HOST to replace DEFAULT_HOST



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: how can i increase size of heap

[Matthias J. Sax]

1) yes, this is correct (just as you do it on command line, ie, multiple
flags separated by blank)

worker childopts : "-Xms4g -Djava.net.preferIPv4Stack=true"

2) still nor sure what you mean... if you get GC overhead limit exceed
it means that the GC tries to clean up objects to free memory, but no
objects could be deleted (and GC tried multiple time without success).
This is a Java thing and not directly related to Storm. You should be
able to tackle it, by increasing the JVM memory, as you already do by (1) :)

-Matthias


On 04/16/2016 01:29 PM, sam mohel wrote:
> thanks but what if i want to write this also in worker childopts :
> "-Djava.net.preferIPv4Stack=true"
> 
> is it will be like
> 
> worker childopts : "-Xms4g -Djava.net.preferIPv4Stack=true" ?
> 
> for Second question i got this statemnent by searching for my problem with
> GC overhead limited exceed but didn't know how can i do it ?
> 
> 
> On Sat, Apr 16, 2016 at 1:22 PM, Matthias J. Sax  wrote:
> 
>> use parameter worker.childopts
>>
>> worker.childopts: "-Xms4g"
>>
>> Not sure what you mean by your second question...
>>
>> -Matthias
>>
>> On 04/16/2016 06:34 AM, sam mohel wrote:
>>> i want to increase size of heap for worker to -Xms4G how can i write it
>> in
>>> storm.yaml ?
>>>
>>> and how can i set the number of CPU  consuming bolts to the amount of
>> cores
>>> ?
>>>
>>> Thanks for any help
>>>
>>
>>
> 



signature.asc
Description: OpenPGP digital signature

Re: BYLAWS link incorrent in DEVELOPER.md

[Matthias J. Sax]

Thanks! I just fixed it.

-Matthias

On 04/16/2016 10:35 AM, Manu Zhang wrote:
> Hi committers,
> 
> FYI, The BYLAWS link
> 
> in DEVELOPER.md document is incorrect and returns 404.
> 
> Thanks,
> Manu Zhang
> 



signature.asc
Description: OpenPGP digital signature

Re: how can i increase size of heap

[sam mohel]

thanks but what if i want to write this also in worker childopts :
"-Djava.net.preferIPv4Stack=true"

is it will be like

worker childopts : "-Xms4g -Djava.net.preferIPv4Stack=true" ?

for Second question i got this statemnent by searching for my problem with
GC overhead limited exceed but didn't know how can i do it ?


On Sat, Apr 16, 2016 at 1:22 PM, Matthias J. Sax  wrote:

> use parameter worker.childopts
>
> worker.childopts: "-Xms4g"
>
> Not sure what you mean by your second question...
>
> -Matthias
>
> On 04/16/2016 06:34 AM, sam mohel wrote:
> > i want to increase size of heap for worker to -Xms4G how can i write it
> in
> > storm.yaml ?
> >
> > and how can i set the number of CPU  consuming bolts to the amount of
> cores
> > ?
> >
> > Thanks for any help
> >
>
>

Re: how can i increase size of heap

[Matthias J. Sax]

use parameter worker.childopts

worker.childopts: "-Xms4g"

Not sure what you mean by your second question...

-Matthias

On 04/16/2016 06:34 AM, sam mohel wrote:
> i want to increase size of heap for worker to -Xms4G how can i write it in
> storm.yaml ?
> 
> and how can i set the number of CPU  consuming bolts to the amount of cores
> ?
> 
> Thanks for any help
> 



signature.asc
Description: OpenPGP digital signature