Re: Set worker Xmx

But does that not set it for all workers? What If I want to the specific? Regards > On 16 Apr 2016, at 14:19, Matthias J. Sax wrote: > > topology.worker.childopts > > On 04/16/2016 02:38 PM, Benjamin Cuthbert wrote: >> All, >> >> How do you set the individual

Set worker Xmx

All, How do you set the individual worker Xmx settings for different topologies.

Re: zookeeper paths not created on startup

This line of code long lastCompletedOffset = lastCompletedOffset(); if (_committedTo != lastCompletedOffset) { In PartitionManager Will always be = in my case it is never different so there is never a write. > On 24 May 2015, at 18:18, Benjamin Cuthbert wrote: > > The

Re: zookeeper paths not created on startup

; use GET > > get /brokers/rawWarehousePriceSpout/partition_0 > > 2015-05-21 15:27 GMT-03:00 Benjamin Cuthbert <mailto:cuthbert@gmail.com>>: > So I do not see what you see. So just so I am sure the SpoutConfig when > deployed into an env should have the zookeeper host

Re: debugging why storm is not writing to zookeeper

Interesting my kafka did not create that it just did /brokers [zk: 127.0.0.1:2181(CONNECTED) 5] ls / [hbase, zookeeper, admin, consumers, config, hive_zookeeper_namespace_hive, controller, storm, brokers, controller_epoch] [zk: 127.0.0.1:2181(CONNECTED) 6] ls /brokers [rawWarehousePriceSpout, top

debugging why storm is not writing to zookeeper

All, I am having trouble with the spouts not writing data to zookeeper. When I startup storm I see the worker log with 2015-05-24T09:23:36.098+ s.k.PartitionManager [WARN] Error reading and/or parsing at ZkNode: /brokers/rawWarehousePriceSpout/partition_2 java.lang.NullPointerException: nul

Re: zookeeper paths not created on startup

configuration. configured start offset time: [-2] 2015-05-22T08:07:19.551+ s.k.PartitionManager [WARN] Using new offset: 2377398 > On 21 May 2015, at 19:28, Cristian Makoto Sandiga wrote: > > use GET > > get /brokers/rawWarehousePriceSpout/partition_0 > > 2015-05-21 15:27 GMT-03

Re: zookeeper paths not created on startup

local: [zk: localhost:2000(CONNECTED) 9] get /brokers/rawWarehousePriceSpout/partition_0 0 cZxid = 0x42 ctime = Thu May 21 19:25:26 BST 2015 mZxid = 0x42 mtime = Thu May 21 19:25:26 BST 2015 pZxid = 0x42 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 1 numChildren =

Re: zookeeper paths not created on startup

> 2015-05-21 11:28 GMT-03:00 Cristian Makoto Sandiga <mailto:cmsand...@gmail.com>>: > I dont know, i'm using BaseRichBolt. > > 2015-05-21 11:25 GMT-03:00 Benjamin Cuthbert <mailto:cuthbert@gmail.com>>: > > I am using BaseBasicBolt as that was told

Re: zookeeper paths not created on startup

r) > > 2015-05-21 11:28 GMT-03:00 Cristian Makoto Sandiga <mailto:cmsand...@gmail.com>>: > I dont know, i'm using BaseRichBolt. > > 2015-05-21 11:25 GMT-03:00 Benjamin Cuthbert <mailto:cuthbert@gmail.com>>: > > I am using BaseBasicBolt as that

Re: zookeeper paths not created on startup

I am using BaseBasicBolt as that was told ack events automatically. Should we not use that? > On 21 May 2015, at 15:24, Cristian Makoto Sandiga wrote: > > Are you acking in your bolt? collector.ack(input); > > 2015-05-21 11:19 GMT-03:00 Benjamin Cuthbert <mailto:cuth

Re: zookeeper paths not created on startup

;id":"a9be1962-6b4e-4ed4-ae68-155a1948a1f6","name":"consolidate_reports"},"offset":4426029,"partition":0,"broker":{"host":"localhost","port":9092},&q

Re: zookeeper paths not created on startup

; conf.get(Config.STORM_ZOOKEEPER_SERVERS); > } > Integer zkPort = _spoutConfig.zkPort; > if (zkPort == null) { > zkPort = ((Number) > conf.get(Config.STORM_ZOOKEEPER_PORT)).intValue(); > } > > > > > 2015-05-21 9:53 GMT-03:00 Benj

Re: zookeeper paths not created on startup

r zkPort = _spoutConfig.zkPort; > if (zkPort == null) { > zkPort = ((Number) > conf.get(Config.STORM_ZOOKEEPER_PORT)).intValue(); > } > > > > > 2015-05-21 9:53 GMT-03:00 Benjamin Cuthbert <mailto:cuthbert@gmail.com>>: > So when I run it

Re: zookeeper paths not created on startup

eSpout/partition_2 --> null. > > Try to run Local Mode, send some messages to spout, and then see Offset in > Zookeeper (port 2000). > > Or use Zookeeper of kafka, override. > > spoutconfig.zkServers > spoutconfig.zkPort > > > 2015-05-21 9:39 GMT-03:00 Benjami

Re: zookeeper paths not created on startup

per create nothing when startup, you have to create your partitions in > kafka broker. > > bin/kafka-topics.sh --create --zookeeper localhost:2181 --topic click_history > --replication-factor 1 --partitions 10 > > > > 2015-05-21 8:58 GMT-03:00 Benjamin Cuthbert <ma

zookeeper paths not created on startup

All, We changed or paths in zookeeper and we are now seeing java.lang.RuntimeException: java.lang.RuntimeException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /brokers/topics/warehouse_prices/partitions at storm.kafka.DynamicBrokersReader.getBrok

Re: kafka offset WARN stops tuples incoming

e offset to 'x+2' for consumer and issue a warning that > the offset 'x+1' is not there. This cycle will continue. > > -Nikhil > > > > On Wednesday, May 20, 2015 3:24 PM, Benjamin Cuthbert > wrote: > > > I am seeing the fol

Re: kafka offset WARN stops tuples incoming

oes it get created automatically? > On 19 May 2015, at 18:19, Benjamin Cuthbert wrote: > > The kafka logs have this > > [2015-05-19 17:13:39,772] ERROR [KafkaApi-0] Error when processing fetch > request for partition [warehouse_prices,0] offset 73792051 from consumer wit

Re: kafka offset WARN stops tuples incoming

14:54, Benjamin Cuthbert wrote: > > Hi Jeff, > > So I looked at the docs and I reset the following property: > > SpoutConfig spoutConfig = new SpoutConfig( > hosts, > t

Re: kafka offset WARN stops tuples incoming

-1.5.1.jar:na] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > On 18 May 2015, at 22:13, Benjamin Cuthbert wrote: > > Thanks Jeff, > > So I looked over the docs but I don’t understand is it runs for 2+ hours then > just starts going: > > 2015

Re: kafka offset WARN stops tuples incoming

ferring to whether you are > going to force starting consumption from a different offset. > " > > Thank you for your time! > > + > Jeff Maass <mailto:maas...@gmail.com> > linkedin.com/in/jeffmaass <http://linkedin.com/in/jeffmaass> > st

kafka offset WARN stops tuples incoming

All, We are getting loads of these errors 2015-05-18T19:52:44.038+0100 s.k.KafkaUtils [WARN] Got fetch request with offset out of range: [62379213]; retrying with default start offset time from configuration. configured start offset time: [-2] 2015-05-18T19:52:44.066+0100 s.k.PartitionManager [

Disconnect / reconnect

All, How does the spout handle disconnect in the event of the below happening. 2015-05-15T11:58:01.589+0100 o.a.s.c.ConnectionState [ERROR] Connection timed out for connection string (1.1.1.1:2181/storm) and timeout (15000) / elapsed (19101) org.apache.storm.curator.CuratorConnectionLossExcepti

Re: logback file in jar

linkedin.com/in/jeffmaass <http://linkedin.com/in/jeffmaass> > stackoverflow.com/users/373418/maassql > <http://stackoverflow.com/users/373418/maassql> > + > > > On Wed, May 13, 2015 at 9:42 AM, Benjamin Cuthbert <mailto:cuthbert@gmail.com>> wrote: > Is there a plan to remove the logback.xml file from the storm-core jar? >

handling Kafka log roll

Does anyone have any suggestions on how to handle the kafka log roll. When this happens my topology just stops processing.

logback file in jar

Is there a plan to remove the logback.xml file from the storm-core jar?

avrò file format with HdfsBolt

Has anyone implement an avro file format with the HdfsBolt org.apache.storm.hdfs.format.RecordFormat

Re: upgrade of disruptor

Thanks Eric, We did attempt an upgrade it did look to work sorta. > On 31 Mar 2015, at 15:18, Eric Ruel wrote: > > https://issues.apache.org/jira/browse/STORM-350 > <https://issues.apache.org/jira/browse/STORM-350>​ > De : Benjamin Cuthbert <mailto:cuthbert@gmail.

upgrade of disruptor

All, is there any chance of an upgrade to the latest disruptor library? Looking at maven central there is http://search.maven.org/#artifactdetails%7Ccom.lmax%7Cdisruptor%7C3.3.2%7Cjar

upgrade of disruptor to 3.20

All, is there any chance of an upgrade to the latest disruptor library? Looking at maven central there is http://search.maven.org/#artifactdetails%7Ccom.lmax%7Cdisruptor%7C3.3.2%7Cjar