The most common reason for spout failures but no bolt failures is tuple timeouts. Try increasing MESSAGE_TIMEOUT_SECONDS. If you are seeing very low execute latencies for your bolts but very high complete latency, you can also try decreasing SPOUT_MAX_PENDING.
Hopefully this helps. On Fri, Dec 9, 2016 at 10:26 PM, pradeep s <sreekumar.prad...@gmail.com> wrote: > Hi, > We are running a 5 node cluster(2x large ec2) with below config. Topology > is consuming message from SQS and writing to RDS and S3. Even if there are > no bolt failures , we are seeing many spout failures. > Can you please help in checking the config.Also i am setting tasks as > parallelism count * 2. Is this fine? > > TOPOLOGY_NAME=MDP_STORM_PRD > MARIA_BOLT_PARALLELISM=50 > S3_BOLT_PARALLELISM=50 > SQS_DELETE_BOLT_PARALLELISM=100 > SPOUT_PARALLELISM=50 > NUMBER_OF_WORKERS=5 > NUMBER_OF_ACKERS=5 > SPOUT_MAX_PENDING=5000 > MESSAGE_TIMEOUT_SECONDS=240 > > > Topology Code > ============= > > Config config = new Config(); > config.setNumWorkers(numWorkers); > config.setDebug(false); > config.setNumAckers(numAckers); > config.setMaxSpoutPending(maxSpoutPending); > config.setMessageTimeoutSecs(messageTimeoutSecs); > > > topologyBuilder.setSpout(spoutId, new > SQSMessageReaderSpout(sqsUtils.getSQSUrl(dataQueue), > properties), > spoutParallelism).*setNumTasks(spoutParallelism * TWO);* > topologyBuilder.setBolt(mariaBoltId, new MariaDbBolt(properties), > mariaBoltParallelism) > .*setNumTasks(mariaBoltParallelism * TWO)*.fieldsGrouping(spoutId, > new Fields(MESSAGE_ID)); > > topologyBuilder.setBolt(s3BoltId, new S3WriteBolt(properties, > s3Properties), s3BoltParallelism) > .*setNumTasks(s3BoltParallelism * TWO).*shuffleGrouping( > mariaBoltId); > > topologyBuilder > .setBolt(sqsDeleteBoltId, new > SQSMessageDeleteBolt(sqsUtils.getSQSUrl(dataQueue)), > sqsBoltParallelism) > .*setNumTasks(sqsBoltParallelism * TWO)*.shuffleGrouping(s3BoltId) > ; > > StormSubmitter.submitTopology(topologyName, config, topologyBuilder. > createTopology()); > > > Regards > > Pradeep S >
