[
https://issues.apache.org/jira/browse/MADLIB-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Orhan Kislal closed MADLIB-1372.
--------------------------------
Resolution: Fixed
> MADlib Keras operations create too many threads
> -----------------------------------------------
>
> Key: MADLIB-1372
> URL: https://issues.apache.org/jira/browse/MADLIB-1372
> Project: Apache MADlib
> Issue Type: Bug
> Components: Deep Learning
> Reporter: Orhan Kislal
> Priority: Major
> Fix For: v1.17
>
>
> We noticed that madlib keras operations that call keras functions create a
> number of threads every single time they are called. However, some of these
> threads are not cleaned up at the end of the function. If the number of
> iterations is very high, the thread count keeps increasing and eventually
> hits the system default limit (1024).
> Here is the error message from the log
> {code}
> what(): Resource temporarily
> unavailable",,,,,,,,"SysLoggerMain","syslogger.c",618,
> {code}
> We tried keeping the session at the end of the function (saved the session
> info in SD) and reuse it for the next iteration but that didn't help with
> this issue.
> It is possible to increase this limit by editing /etc/security/limits.conf
> and the files in /etc/security/limits.d/. This requires a restart of the
> system and the database to take effect.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
