Hi Christopher, you might want to turn off `watch_tools` if you are using it, this is only needed when using the pretty-much undocumented feature where you dump tools into a directory without registering them in any tool_conf.xml file. All other tool reloading happens by another means.
Similarly, if you're using 19.05 and you don't manipulate the tool data table entries manually you can disable `watch_tool_data_dir`. Finally this is usually a sign of allocating too many processes and threads for your webservers/handlers/mules, as each one starts their own watchdog. A common misconception is that this makes Galaxy run faster. but it actually just increases the amount of concurrent requests Galaxy can handle. Best, Marius On Wed, 10 Jul 2019 at 11:43, Previti < christopher.prev...@dkfz-heidelberg.de> wrote: > Dear all, > > I've been getting this error for a few days now, it basically prevents > anything from being installed on the first go. > Individual tools can be installed, but even then I have to try multiple > times. > > Any ideas how to fix this? > > Cheers, > Christopher > > The details from galaxy.log are (installing an older VCFtools package): > > 172.22.24.119 - - [10/Jul/2019:10:44:32 +0200] "POST > /admin_toolshed/prepare_for_install HTTP/1.1" 200 - > " > http://dkfzgalaxy.inet.dkfz-heidelberg.de/admin_toolshed/prepare_for_install?change > > set_revisions=34a6b690e4b5&repository_ids=2ba42187f2588c4b&tool_shed_url=https%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F > <http://dkfzgalaxy.inet.dkfz-heidelberg.de/admin_toolshed/prepare_for_install?changeset_revisions=34a6b690e4b5&repository_ids=2ba42187f2588c4b&tool_shed_url=https%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F> > " > "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:67.0) Gecko/20100101 > Firefox/67.0" > galaxy.tools.toolbox.base DEBUG 2019-07-10 10:44:55,592 Appending to > tool panel section: NGS:VCFtools > 172.22.24.119 - - [10/Jul/2019:10:44:58 +0200] "POST > /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - > "http://dkfzgalaxy.inet.dkfz-heidelberg.de/admin_toolshed/prepare > _for_install" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:67.0) > Gecko/20100101 Firefox/67.0" > 172.22.24.119 - - [10/Jul/2019:10:45:01 +0200] "POST > /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - > "http://dkfzgalaxy.inet.dkfz-heidelberg.de/admin_toolshed/prepare > _for_install" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:67.0) > Gecko/20100101 Firefox/67.0" > 172.22.24.119 - - [10/Jul/2019:10:44:55 +0200] "POST > /admin_toolshed/install_repositories HTTP/1.1" 500 - > " > http://dkfzgalaxy.inet.dkfz-heidelberg.de/admin_toolshed/prepare_for_install > " > "Moz > illa/5.0 (X11; Fedora; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0" > *Error - <type 'exceptions.OSError'>: inotify instance limit reached* > URL: > > http://dkfzgalaxy.inet.dkfz-heidelberg.de/admin_toolshed/install_repositories > File '/opt/galaxy/galaxy/lib/galaxy/web/framework/middleware/error.py', > line 154 in __call__ > app_iter = self.application(environ, sr_checker) > File > '/opt/galaxy/galaxy/.venv/lib/python2.7/site-packages/paste/recursive.py', > line 85 in __call__ > return self.application(environ, start_response) > File > > '/opt/galaxy/galaxy/.venv/lib/python2.7/site-packages/paste/httpexceptions.py', > line 640 in __call__ > return self.application(environ, start_response) > File '/opt/galaxy/galaxy/lib/galaxy/web/framework/base.py', line 143 in > __call__ > return self.handle_request(environ, start_response) > File '/opt/galaxy/galaxy/lib/galaxy/web/framework/base.py', line 222 in > handle_request > body = method(trans, **kwargs) > File '/opt/galaxy/galaxy/lib/galaxy/web/framework/decorators.py', line > 101 in decorator > return func(self, trans, *args, **kwargs) > File > > '/opt/galaxy/galaxy/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', > line 575 in install_repositories > reinstalling=reinstalling, > File > '/opt/galaxy/galaxy/lib/tool_shed/galaxy_install/install_manager.py', > line 846 in install_repositories > tool_panel_section_mapping=tool_panel_section_mapping) > File > '/opt/galaxy/galaxy/lib/tool_shed/galaxy_install/install_manager.py', > line 893 in install_tool_shed_repository > tool_panel_section_mapping=tool_panel_section_mapping) > File > '/opt/galaxy/galaxy/lib/tool_shed/galaxy_install/install_manager.py', > line 544 in __handle_repository_contents > repository_tools_tups = irmm.get_repository_tools_tups() > File > > '/opt/galaxy/galaxy/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py', > line 75 in get_repository_tools_tups > tool = self.app.toolbox.load_tool(os.path.abspath(load_relative_path), > guid=guid, use_cached=False) > File '/opt/galaxy/galaxy/lib/galaxy/tools/toolbox/base.py', line 766 in > load_tool > self.watch_tool(tool) > File '/opt/galaxy/galaxy/lib/galaxy/tools/toolbox/base.py', line 774 in > watch_tool > self._tool_watcher.watch_file(tool.config_file, tool.id) > File '/opt/galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py', line 198 > in watch_file > self.monitor(tool_dir) > File '/opt/galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py', line 190 > in monitor > self.observer.schedule(self.event_handler, dir, recursive=False) > File > > '/opt/galaxy/galaxy/.venv/lib/python2.7/site-packages/watchdog/observers/api.py', > line 293 in schedule > emitter.start() > File > > '/opt/galaxy/galaxy/.venv/lib/python2.7/site-packages/watchdog/utils/__init__.py', > line 110 in start > self.on_thread_start() > File > > '/opt/galaxy/galaxy/.venv/lib/python2.7/site-packages/watchdog/observers/inotify.py', > line 121 in on_thread_start > self._inotify = InotifyBuffer(path, self.watch.is_recursive) > File > > '/opt/galaxy/galaxy/.venv/lib/python2.7/site-packages/watchdog/observers/inotify_buffer.py', > line 35 in __init__ > self._inotify = Inotify(path, recursive) > File > > '/opt/galaxy/galaxy/.venv/lib/python2.7/site-packages/watchdog/observers/inotify_c.py', > line 188 in __init__ > Inotify._raise_error() > *File > > '/opt/galaxy/galaxy/.venv/lib/python2.7/site-packages/watchdog/observers/inotify_c.py', > line 415 in _raise_error** > ** raise OSError("inotify instance limit reached")** > **OSError: inotify instance limit reached** > * > CGI Variables > ------------- > CONTENT_LENGTH: '12169' > CONTENT_TYPE: 'application/x-www-form-urlencoded; charset=UTF-8' > HTTP_ACCEPT: 'text/html, */*; q=0.01' > HTTP_ACCEPT_ENCODING: 'gzip,deflate' > HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.5' > HTTP_CONNECTION: 'Keep-Alive' > HTTP_COOKIE: > > 'galaxysession=c6ca0ddb55be603a9d01bef828912a83dd6a820b8a47ad278d223920b71ba8b43726aaf1140b008e' > HTTP_HOST: 'dkfzgalaxy.inet.dkfz-heidelberg.de' > HTTP_REFERER: > ' > http://dkfzgalaxy.inet.dkfz-heidelberg.de/admin_toolshed/prepare_for_install > ' > HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:67.0) > Gecko/20100101 Firefox/67.0' > HTTP_X_FORWARDED_FOR: '172.22.24.119' > HTTP_X_FORWARDED_HOST: 'dkfzgalaxy.inet.dkfz-heidelberg.de' > HTTP_X_FORWARDED_SERVER: 'fe80::216:3eff:fe80:c02b' > HTTP_X_REQUESTED_WITH: 'XMLHttpRequest' > ORGINAL_HTTP_HOST: 'localhost:8080' > ORGINAL_REMOTE_ADDR: '127.0.0.1' > PATH_INFO: '/admin_toolshed/install_repositories' > REMOTE_ADDR: '172.22.24.119' > REQUEST_METHOD: 'POST' > SERVER_NAME: '127.0.0.1' > SERVER_PORT: '8080' > SERVER_PROTOCOL: 'HTTP/1.1' > > > WSGI Variables > -------------- > application: <paste.recursive.RecursiveMiddleware object at > 0x7f9f48207410> > controller_action_key: u'web.admin_toolshed.install_repositories' > is_api_request: False > paste.cookies: (<SimpleCookie: > > galaxysession='c6ca0ddb55be603a9d01bef828912a83dd6a820b8a47ad278d223920b71ba8b43726aaf1140b008e'>, > > 'galaxysession=c6ca0ddb55be603a9d01bef828912a83dd6a820b8a47ad278d223920b71ba8b43726aaf1140b008e') > paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] > paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler > object at 0x7f9f48c86fd0> > paste.httpserver.proxy.host: 'dummy' > paste.httpserver.proxy.scheme: 'http' > paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at > 0x7f9f486fcd90> > paste.recursive.forward: <paste.recursive.Forwarder from /> > paste.recursive.include: <paste.recursive.Includer from /> > paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from > /> > paste.recursive.script_name: '' > paste.throw_errors: True > request_id: 'fd8cc98ea2ee11e9a89100163e80c02b' > webob._body_file: (<_io.BufferedReader>, <socket._fileobject object at > 0x7f9f3a4b5ad0 length=12169>) > webob._parsed_post_vars: (MultiDict([('tool_shed_repository_ids', > u"[u'ac218635d795e337', u'e3e09dfaa8fad8c8', u'1efe9eb0bff40152', > u'358d8704ef89cb6b', u'cbc5119f47881258']"), ('encoded_kwd', > > u'59b03a7799b953da9827424b9cef2236c5b9f461:7b22737461747573223a2022646f6e65222c20226861735f7265706f7369746f72795f646570656e64656e63696573223a2066616c73652c2022696e636c756465735f746f6f6c735f666f725f646973706c61795f696e5f746f6f6c5f70616e656c223a20747275652c2022746f6f6c5f736865645f7265706f7369746f72795f696473223a205b2261633231383633356437393565333337222c202265336530396466616138666164386338222c202231656665396562306266663430313532222c202233353864383730346566383963623662222c202263626335313139663437383831323538225d2c2022736865645f746f6f6c5f636f6e66223a20222e2f636f6e6669672f736865645f746f6f6c5f636f6e662e786d6c222c2022696e7374616c6c5f7265706f7369746f72795f646570656e64656e63696573223a20747275652c2022746f6f6c5f70617468223a20222f6f70742f67616c6178792f736865645f746f6f6c73222c20227265706f5f696e666f5f6469637473223a2... > 0x7f9f35cdfed0>) > webob._parsed_query_vars: (GET([]), '') > webob.is_body_seekable: True > wsgi process: 'Multithreaded' > wsgi.input_terminated: True > > > > > -- > *Dr. Christopher Previti* > Genomics and Proteomics Core Facility > High Throughput Sequencing (W190) > Bioinformatician > > German Cancer Research Center (DKFZ) > Foundation under Public Law > Im Neuenheimer Feld 580 > 69120 Heidelberg > Germany > Room: B2.102 (INF580/TP3) > Phone: +49 6221 42-4661 > > christopher.prev...@dkfz.de <http://www.dkfz.de/> > www.dkfz.de <http://www.dkfz.de/> > > Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta > VAT-ID No.: DE143293537 > > Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die > Personen bestimmt, an die sie adressiert ist. > Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte > Informationen enthalten. Sollten Sie nicht > der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den > Absender und löschen Sie die Mitteilung. > Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist > untersagt. > > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > %(web_page_url)s > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/ ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: %(web_page_url)s To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
