*Hi,*
*I have a bolt which emits around 15000 tuples sometimes. Sometimes it
emits more than 20000 tuples. I think when this happens, there's a memory
issue and the workers get restarted. This is what worker.log.err contains:*
*Java HotSpot(TM) 64-Bit Server VM warning: INFO:
os::commit_memory(0x00000000f1000000, 62914560, 0) failed; error='Cannot
allocate memory' (errno=12)# There is insufficient memory for the Java
Runtime Environment to continue.# Native memory allocation (mmap) failed to
map 62914560 bytes for committing reserved memory.# An error report file
with more information is saved as:#
/home/storm/apache-storm-1.0.0/storm-local/workers/6a1a70ad-d094-437a-a9c5-e837fc1b3535/hs_err_pid2766.log*
*The odd part is, that in all my bolts I have *
* @Override public void execute(Tuple tuple) { try {*
*..some code; including the code that emits tuples*
*} catch(Exception ex) {logger.info <http://logger.info>("The exception {},
{}", ex.getCause(), ex.getMessage());} }*
*But in the logs I never see the string "The exception". But worker.log
shows:*
*2017-02-05 09:14:01.320 STDERR [INFO] Java HotSpot(TM) 64-Bit Server VM
warning: INFO: os::commit_memory(0x00000000e6f80000, 37748736, 0) failed;
error='Cannot allocate memory' (errno=12)2017-02-05 09:14:01.320 STDERR
[INFO] #2017-02-05 09:14:01.330 STDERR [INFO] # There is insufficient
memory for the Java Runtime Environment to continue.2017-02-05 09:14:01.330
STDERR [INFO] # Native memory allocation (mmap) failed to map 37748736
bytes for committing reserved memory.2017-02-05 09:14:01.331 STDERR [INFO]
# An error report file with more information is saved as:2017-02-05
09:14:01.331 STDERR [INFO] #
/home/storm/apache-storm-1.0.0/storm-local/workers/2685b445-c4a9-4f7e-94e1-1ce3fe13de47/hs_err_pid3022.log2017-02-05
09:14:06.904 o.a.s.d.worker [INFO] Launching worker for
HydraCellGen-138-1486283223 on 3fc3c05e-9769-4033-bf7d-df609d6c4963:6701
with id 575bd7ed-a3fc-4f7f-a7d0-cdd4054c9fc5 and conf
{"topology.builtin.metrics.bucket.size.secs" 60, "nimbus.childopts"
"-Xmx1024m",... etc*
*These are the settings I'm using for the topology:*
* Config stormConfig = new Config();
stormConfig.setNumWorkers(20); stormConfig.setNumAckers(20);
stormConfig.put(Config.TOPOLOGY_DEBUG, false);
stormConfig.put(Config.TOPOLOGY_TRANSFER_BUFFER_SIZE,
1024); stormConfig.put(Config.TOPOLOGY_EXECUTOR_RECEIVE_BUFFER_SIZE,
65536);
stormConfig.put(Config.TOPOLOGY_EXECUTOR_SEND_BUFFER_SIZE,
65536); stormConfig.put(Config.TOPOLOGY_MAX_SPOUT_PENDING,
2); stormConfig.put(Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS,
2200); stormConfig.put(Config.STORM_ZOOKEEPER_SERVERS,
Arrays.asList(new String[]{"localhost"}));
stormConfig.put(Config.TOPOLOGY_WORKER_CHILDOPTS, "-Xmx" + "2g");*
*So am I right in assuming the exception is not thrown in my code but is
thrown in the worker thread? Do such exceptions happen when the worker
isn't able to receive too many tuples in its queue?*
*What can I do to avoid this problem?*
--
Regards,
Navin
