[INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building storm-starter 0.9.3-incubating-SNAPSHOT [INFO] ------------------------------------------------------------------------ [WARNING] The POM for org.apache.storm:storm-core:jar:0.9.3-incubating is missin g, no dependency information available [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Storm .............................................. SUCCESS [ 2.816 s] [INFO] maven-shade-clojure-transformer .................... SUCCESS [ 30.954 s] [INFO] Storm Core ......................................... SUCCESS [01:34 min] [INFO] storm-starter ...................................... FAILURE [ 0.128 s] [INFO] storm-kafka ........................................ SKIPPED [INFO] storm-hdfs ......................................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 02:08 min [INFO] Finished at: 2014-09-16T23:51:57+02:00 [INFO] Final Memory: 39M/411M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal on project storm-starter: Could not resolve depen dencies for project org.apache.storm:storm-starter:jar:0.9.3-incubating-SNAPSHOT : Failure to find org.apache.storm:storm-core:jar:0.9.3-incubating in http://rep o1.maven.org/maven2/ was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forc ed -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e swit ch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please rea d the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyReso lutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :storm-starter On Tue, Sep 16, 2014 at 11:46 PM, Harsha <st...@harsha.io> wrote: > Hi Alec, > Single node kafka cluster not recommended apart from using it for > development. I highly recommend using multinode cluster and create a > partitioned topic with replication. This not only makes it optimal to take > in more data at faster rates also allows your cluster running if there is a > node failure as the topic is replicated there wouldn't be huge data loss. > > " If I am using multiple-nodes, the tradeoff is the connection time among > different nodes?" > kafka producer api sends a message to broker either round-robin > or based on partition function. > please go through the kafka docs here > http://kafka.apache.org/documentation.html for simple consumer and also > how the replication works among multiple nodes. > > -Harsha > > > On Tue, Sep 16, 2014, at 02:06 PM, Sa Li wrote: > > Hi, All > > I have been using kafka cluster in single server with three brokers, but I > am thinking to build a larger kafka cluster, say 4 nodes (server), and 3 > brokers in each node, so totally 12 brokers, would that be better than > single node cluster? Or single node will be fair enough, since web api may > push million rows into kafka cluster every day, I am kinda worry if the > cluster is capable to take such much data without losing data. If I am > using multiple-nodes, the tradeoff is the connection time among different > nodes? > > > thanks > > Alec > > >
