Re: Storm Benchmarks

Thank you Anshu. I tried that but it doesn't work. Same error: Exception in thread "main" java.lang.NoClassDefFoundError: backtype/storm/generated/AlreadyAliveException. Regards WA From: anshu shukla To: u...@storm.apache.org Cc: Walid Aljoby

[jira] [Commented] (STORM-1977) Leader Nimbus crashes with getClusterInfo when it doesn't have one or more replicated topology codes

[ https://issues.apache.org/jira/browse/STORM-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383386#comment-15383386 ] ASF GitHub Bot commented on STORM-1977: --- Github user HeartSaVioR commented on the issue:

[GitHub] storm issue #1574: STORM-1977 Restore logic: give up leadership when elected...

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1574 @revans2 Let me back to code change. Could you clarify that this change can also work well with HDFS backed BlobStore? If not how to do this? I'd like to address this as abstraction

[jira] [Commented] (STORM-1977) Leader Nimbus crashes with getClusterInfo when it doesn't have one or more replicated topology codes

[ https://issues.apache.org/jira/browse/STORM-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383353#comment-15383353 ] ASF GitHub Bot commented on STORM-1977: --- Github user HeartSaVioR commented on the issue:

[GitHub] storm issue #1574: STORM-1977 Restore logic: give up leadership when elected...

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1574 @revans2 I'm just thinking about the responsibility of Nimbus. - Nimbus was "soft" SPOF and we claimed that Nimbus is designed to fail-fast and stateless so just supervising Nimbus

ApacheCon: Getting the word out internally

ApacheCon: Getting the word out internally Dear Apache Enthusiast, As you are no doubt already aware, we will be holding ApacheCon in Seville, Spain, the week of November 14th, 2016. The call for papers (CFP) for this event is now open, and will remain open until September 9th. The event is

[jira] [Updated] (STORM-1981) storm command hangs

[ https://issues.apache.org/jira/browse/STORM-1981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghav Kumar Gautam updated STORM-1981: --- Description: I ran the following command which threw an exception and never

[jira] [Created] (STORM-1981) storm command hangs

Raghav Kumar Gautam created STORM-1981: -- Summary: storm command hangs Key: STORM-1981 URL: https://issues.apache.org/jira/browse/STORM-1981 Project: Apache Storm Issue Type: Bug

[jira] [Commented] (STORM-1977) Leader Nimbus crashes with getClusterInfo when it doesn't have one or more replicated topology codes

[ https://issues.apache.org/jira/browse/STORM-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383134#comment-15383134 ] ASF GitHub Bot commented on STORM-1977: --- Github user harshach commented on the issue:

[GitHub] storm issue #1574: STORM-1977 Restore logic: give up leadership when elected...

Github user harshach commented on the issue: https://github.com/apache/storm/pull/1574 +1. Agree on having a discussion around these scenarios in another JIRA. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[jira] [Commented] (STORM-1976) Storm Nimbus H/A has issue on cleaning corrupted topologies

[ https://issues.apache.org/jira/browse/STORM-1976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383132#comment-15383132 ] ASF GitHub Bot commented on STORM-1976: --- Github user harshach commented on the issue:

[GitHub] storm issue #1572: STORM-1976 Remove cleanup-corrupt-topologies! (1.x)

Github user harshach commented on the issue: https://github.com/apache/storm/pull/1572 @HeartSaVioR @revans2 are we ok with merging this in and file a follow-up JIRA to have a discussion around the HA and expectations from user. --- If your project is set up for it, you can reply to

[jira] [Commented] (STORM-1930) Kafka New Client API - Support for Topic Wildcards

[ https://issues.apache.org/jira/browse/STORM-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382866#comment-15382866 ] ASF GitHub Bot commented on STORM-1930: --- Github user nabhanelrahman closed the pull request at:

[GitHub] storm pull request #1518: re-submitting STORM-1930: Kafka New Client API - S...

Github user nabhanelrahman closed the pull request at: https://github.com/apache/storm/pull/1518 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Created] (STORM-1980) Command line arguments are not provided to shell/thrift topologies

Darren Warner created STORM-1980: Summary: Command line arguments are not provided to shell/thrift topologies Key: STORM-1980 URL: https://issues.apache.org/jira/browse/STORM-1980 Project: Apache

[jira] [Commented] (STORM-1971) HDFS Timed Synchronous Policy

[ https://issues.apache.org/jira/browse/STORM-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382556#comment-15382556 ] ASF GitHub Bot commented on STORM-1971: --- Github user darionyaphet commented on a diff in the pull

[GitHub] storm pull request #1566: [STORM-1971] HDFS Timed Synchronous Policy

Github user darionyaphet commented on a diff in the pull request: https://github.com/apache/storm/pull/1566#discussion_r71182808 --- Diff: external/storm-hdfs/src/main/java/org/apache/storm/hdfs/bolt/sync/CountSyncPolicy.java --- @@ -26,7 +26,7 @@ * have been processed.

Re: Storm Benchmarks

I don't think storm-benchmark can be run against storm 1.0.1 without code changes. Yes, both computation and communication parts are included. The metrics are collected by storm nimbus including end-to-end throughput and latency. I know nothing more than you on that micro-benchmark. Maybe +

[GitHub] storm issue #1572: STORM-1976 Remove cleanup-corrupt-topologies! (1.x)

Github user revans2 commented on the issue: https://github.com/apache/storm/pull/1572 @harshach I don't think that would fix the issue, although it is an improvement. If I think about it more I think what happened was that they were not running with HA/expecting HA to work, but ran

[jira] [Commented] (STORM-1976) Storm Nimbus H/A has issue on cleaning corrupted topologies

[ https://issues.apache.org/jira/browse/STORM-1976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382537#comment-15382537 ] ASF GitHub Bot commented on STORM-1976: --- Github user revans2 commented on the issue:

[jira] [Commented] (STORM-1976) Storm Nimbus H/A has issue on cleaning corrupted topologies

[ https://issues.apache.org/jira/browse/STORM-1976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382524#comment-15382524 ] ASF GitHub Bot commented on STORM-1976: --- Github user harshach commented on the issue:

[jira] [Commented] (STORM-1977) Leader Nimbus crashes with getClusterInfo when it doesn't have one or more replicated topology codes

[ https://issues.apache.org/jira/browse/STORM-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382505#comment-15382505 ] ASF GitHub Bot commented on STORM-1977: --- Github user revans2 commented on the issue:

[GitHub] storm issue #1574: STORM-1977 Restore logic: give up leadership when elected...

Github user revans2 commented on the issue: https://github.com/apache/storm/pull/1574 @HeartSaVioR I am OK with this change, like I am OK with the change for STORM-1976. I just don't think that this is the final solution for the local blobstore+nimbus, nor do I think that

[jira] [Commented] (STORM-1976) Storm Nimbus H/A has issue on cleaning corrupted topologies

[ https://issues.apache.org/jira/browse/STORM-1976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382439#comment-15382439 ] ASF GitHub Bot commented on STORM-1976: --- Github user HeartSaVioR commented on the issue:

[GitHub] storm issue #1572: STORM-1976 Remove cleanup-corrupt-topologies! (1.x)

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1572 @revans2 Yes agreed. After removing this code we can't remove corrupted topology in normal way since no nimbus can gain leadership. It would be nice to have a tool to show replication count

[GitHub] storm issue #1574: STORM-1977 Restore logic: give up leadership when elected...

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1574 @revans2 And local BlobStore should be designed to achieve hive availability just same as HDFS BlobStore. But the process BlobStore is behind is Nimbus, which is designed to fail-fast,

[jira] [Commented] (STORM-1977) Leader Nimbus crashes with getClusterInfo when it doesn't have one or more replicated topology codes

[ https://issues.apache.org/jira/browse/STORM-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382405#comment-15382405 ] ASF GitHub Bot commented on STORM-1977: --- Github user HeartSaVioR commented on the issue:

[jira] [Commented] (STORM-1977) Leader Nimbus crashes with getClusterInfo when it doesn't have one or more replicated topology codes

[ https://issues.apache.org/jira/browse/STORM-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382375#comment-15382375 ] ASF GitHub Bot commented on STORM-1977: --- Github user HeartSaVioR commented on the issue:

[GitHub] storm issue #1574: STORM-1977 Restore logic: give up leadership when elected...

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1574 @revans2 If we can guarantee that leader nimbus has all topologies code, leader nimbus can function well. Old code is based on this assumption, not considering followers and also not about

[jira] [Commented] (STORM-1976) Storm Nimbus H/A has issue on cleaning corrupted topologies

[ https://issues.apache.org/jira/browse/STORM-1976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382351#comment-15382351 ] Robert Joseph Evans commented on STORM-1976: For me I keep seeing this. Min replication count

[jira] [Commented] (STORM-1976) Storm Nimbus H/A has issue on cleaning corrupted topologies

[ https://issues.apache.org/jira/browse/STORM-1976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382337#comment-15382337 ] ASF GitHub Bot commented on STORM-1976: --- Github user revans2 commented on the issue:

[GitHub] storm issue #1572: STORM-1976 Remove cleanup-corrupt-topologies! (1.x)

Github user revans2 commented on the issue: https://github.com/apache/storm/pull/1572 +1, but I would like to see a tool that can kill a corrupted topology without nimbus being involved, or instructions somewhere on how to do it by logging directly into ZooKeeper. --- If your

[GitHub] storm pull request #1549: Fix Comment Error

Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/1549 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (STORM-1977) Leader Nimbus crashes with getClusterInfo when it doesn't have one or more replicated topology codes

[ https://issues.apache.org/jira/browse/STORM-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382316#comment-15382316 ] Robert Joseph Evans commented on STORM-1977: The entire procedure to reproduce the issue is

[jira] [Commented] (STORM-1971) HDFS Timed Synchronous Policy

[ https://issues.apache.org/jira/browse/STORM-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382308#comment-15382308 ] ASF GitHub Bot commented on STORM-1971: --- Github user dossett commented on a diff in the pull

[GitHub] storm pull request #1566: [STORM-1971] HDFS Timed Synchronous Policy

Github user dossett commented on a diff in the pull request: https://github.com/apache/storm/pull/1566#discussion_r71151878 --- Diff: external/storm-hdfs/src/main/java/org/apache/storm/hdfs/bolt/sync/CountSyncPolicy.java --- @@ -26,7 +26,7 @@ * have been processed. */

[GitHub] storm issue #1549: Fix Comment Error

Github user dossett commented on the issue: https://github.com/apache/storm/pull/1549 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Commented] (STORM-1971) HDFS Timed Synchronous Policy

[ https://issues.apache.org/jira/browse/STORM-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382300#comment-15382300 ] Aaron Dossett commented on STORM-1971: -- [~darion] There is a default value for the tick tuples, 15

[jira] [Commented] (STORM-1977) Leader Nimbus crashes with getClusterInfo when it doesn't have one or more replicated topology codes

[ https://issues.apache.org/jira/browse/STORM-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382295#comment-15382295 ] ASF GitHub Bot commented on STORM-1977: --- Github user revans2 commented on the issue:

[GitHub] storm issue #1574: STORM-1977 Restore logic: give up leadership when elected...

Github user revans2 commented on the issue: https://github.com/apache/storm/pull/1574 The code to do the leader election based off of the IDs was removed because architecturally the blobstore should be in charge of maintaining high availability. Exposing what is stored on the local

[jira] [Updated] (STORM-1917) Storm Redis TTL Control

[ https://issues.apache.org/jira/browse/STORM-1917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Dossett updated STORM-1917: - Component/s: storm-hdfs > Storm Redis TTL Control > --- > >

[jira] [Updated] (STORM-1917) Storm Redis TTL Control

[ https://issues.apache.org/jira/browse/STORM-1917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Dossett updated STORM-1917: - Component/s: (was: storm-hdfs) > Storm Redis TTL Control > --- > >

[jira] [Resolved] (STORM-1909) Fix errors in HDFS Spout documentation

[ https://issues.apache.org/jira/browse/STORM-1909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Dossett resolved STORM-1909. -- Resolution: Fixed Fix Version/s: 1.0.2 > Fix errors in HDFS Spout documentation >

[jira] [Commented] (STORM-634) Storm should support rolling upgrade/downgrade of storm cluster.

[ https://issues.apache.org/jira/browse/STORM-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382291#comment-15382291 ] ASF GitHub Bot commented on STORM-634: -- Github user revans2 commented on the issue:

[GitHub] storm issue #414: STORM-634: Storm serialization changed to thrift to suppor...

Github user revans2 commented on the issue: https://github.com/apache/storm/pull/414 @danny0405 like @HeartSaVioR said it depends on the versions you are upgrading between. Most of the time we have maintained wire and binary compatibility so you can do the upgrade piecemeal. This

Re: Storm Benchmarks

Hi Manu Zhang, Thank you for your prompt replies. I am using Apache storm 1.0.1. So I don't whether it is feasible to change the version of storm inside pom.xml file of storm-benchmark to 1.0.1. ?Do the metrics you use include both computation and communication parts?Do you have an idea about

Re: Storm Benchmarks

Hey Walid, storm-benchmark only supports storm 0.9.x for now. Which storm version do you want to run against ? Regards, Manu Zhang On Mon, Jul 18, 2016 at 7:37 PM Walid Aljoby wrote: > Thank you. > You mean that it could not be run with new version of Apache

Re: Storm Benchmarks

Thank you. You mean that it could not be run with new version of Apache storm? Regards Walid From: darion.yaphet To: dev@storm.apache.org; Walid Aljoby Cc: User Sent: Monday, July 18, 2016 6:35 PM Subject: Re:Storm

Re:Available Metrices

Hi : I'm not very sure what is available network and application metrics ? Is all kind of metics ? 在 2016-07-18 12:01:45，"Walid Aljoby" 写道： Hi Everyone, Anyone can please list the available network and application metrics in storm? It is good if there is specific

Re:Storm Benchmarks

It seems not found backtype.storm.generated.AlreadyAliveException .Because AlreadyAliveException have move to org.apache.storm.generated At 2016-07-18 18:06:36, "Walid Aljoby" wrote: >Hi Everyone, >Anyone used to run Storm Benchmarks in this site: 

Storm Benchmarks

Hi Everyone, Anyone used to run Storm Benchmarks in this site:  https://github.com/roshannaik/storm-benchmark-sol ?The compilation was successful, but I got into error through run time? .0.1/bin/storm -jar ./target/storm-benchmark-0.1.0-jar-with-dependencies.jar -conf ./conf/sol.yaml -c

[jira] [Commented] (STORM-634) Storm should support rolling upgrade/downgrade of storm cluster.

[ https://issues.apache.org/jira/browse/STORM-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382004#comment-15382004 ] ASF GitHub Bot commented on STORM-634: -- Github user HeartSaVioR commented on the issue:

[GitHub] storm issue #414: STORM-634: Storm serialization changed to thrift to suppor...

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/414 Hi @danny0405, could you describe which version do you use and which version do you want to upgrade? --- If your project is set up for it, you can reply to this email and have your reply appear

Re: [VOTE] Release Apache Storm 1.1.0 (rc1)

For leaving history: IMO we should hold on releasing 1.1.0 RC1 for the same reason on 1.0.2 RC3. Nimbus H/A has several issues which seems to be raised while adding BlobStore feature, and I filed STORM-1976 and STORM-1977

[jira] [Commented] (STORM-634) Storm should support rolling upgrade/downgrade of storm cluster.

[ https://issues.apache.org/jira/browse/STORM-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381936#comment-15381936 ] ASF GitHub Bot commented on STORM-634: -- Github user danny0405 commented on the issue:

[jira] [Commented] (STORM-1975) Support default value for KafkaBolt

[ https://issues.apache.org/jira/browse/STORM-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381777#comment-15381777 ] ASF GitHub Bot commented on STORM-1975: --- Github user darionyaphet commented on the issue:

[GitHub] storm issue #1570: [STORM-1975] Support default value for KafkaBolt

Github user darionyaphet commented on the issue: https://github.com/apache/storm/pull/1570 @harshach `Storm-kafka-client` is a new spout to consume data from Kafka . It seems not include `KafkaBolt`. Is it good to port `Storm-kafka`'s `KafkaBolt` into `Storm-kafka-client` ? ---