[
https://issues.apache.org/jira/browse/QPID-8152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16424201#comment-16424201
]
Alex Rudyy commented on QPID-8152:
----------------------------------
The upper limit of the number of inotify instances per real user ID is set to
128 by default on many linux based operation systems.
$ sysctl fs.inotify.max_user_instances
fs.inotify.max_user_instances = 128
Each BDB virtual host creates its own inotify service for detection of
unexpectedly deleted files. Thus, if a number of virtual hosts on all brokers
in the environment exceed 128, the issue would manifest on start-up of 129th
virtual host and the rest of remaining virtual hosts.
The number of max_user_instances can be raised if required. Alternatively, BDB
JE watchers can be disabled by running brokers with
-Dje.log.detectFileDelete=false. The latter will not have any effect on broker
functionality. The broker with watchers disabled would continue to operate as
before. Detection of unexpectedly deleted files is an auxiliary feature which
would allow failing early than later. The bdb je files should not be ever
deleted manually in production systems (apart from the cases when data is
restored from backup or file system gets corrupted).
> [Broker-J][BDB] Disable BDB JE file deletion watcher by default in order to
> avoid running into "user inotify limit"
> --------------------------------------------------------------------------------------------------------------------
>
> Key: QPID-8152
> URL: https://issues.apache.org/jira/browse/QPID-8152
> Project: Qpid
> Issue Type: Improvement
> Components: Broker-J
> Affects Versions: qpid-java-broker-7.0.3
> Reporter: Alex Rudyy
> Priority: Major
>
> BDB JE use registers watcher service to detect unexpected deletion of the
> file.
> In some purely configured environments the user inotify limit can be breached
> on virtual host/virtual host node start-up. This situation ends-up in
> exception like the one below:
> {noformat}
> Caused by: java.io.IOException: User limit of inotify instances reached or
> too many open files
> at
> sun.nio.fs.LinuxWatchService.<init>(LinuxWatchService.java:64)
> at
> sun.nio.fs.LinuxFileSystem.newWatchService(LinuxFileSystem.java:47)
> at
> com.sleepycat.je.log.FileDeletionDetector.<init>(FileDeletionDetector.java:85)
> ... 29 common frames omitted
> {noformat}
> Thought the root of the problem is a poorly configured environment, it would
> be safer to disable file deletion detection watcher by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
