Hello Vaibhav, Did you try setting memory options for Knox ? i.e. setting APP_MEM_OPTS in gateway.sh.
Sometimes, in cases of large memory GC does not run as frequently as it should. Best Sandeep On Tue, Feb 6, 2018 at 6:49 AM, Vaibhav Kulkarni <[email protected]> wrote: > Hi, > > > I am using Knox server gateway (version 0.12.0) with Hortonworks > HDP-2.6.2.0 . There seems to be a memory leak with the knox process > because of which it keeps acquiring more and more memory. My test workload > does very frequent webhdfs rest apis from hundreds of connections. When the > workload is executing, I notice that the knox gateway process memory keeps > increasing. > > > > PID VSZ SZ RSS %CPU TIME CMD > > 24703 44988224 11247056 2195740 78.8 01:37:02 > /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native > -jar /usr/hdp/current/knox-server/bin/ga > > teway.jar > > 24703 44988224 11247056 2195740 78.8 01:37:03 > /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native > -jar /usr/hdp/current/knox-server/bin/ga > > teway.jar > > 24703 44989420 11247355 2335764 84.3 01:44:13 > /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native > -jar /usr/hdp/current/knox-server/bin/ga > > teway.jar > > 24703 44989576 11247394 2326344 89.5 01:51:04 > /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native > -jar /usr/hdp/current/knox-server/bin/ga > > teway.jar > > 24703 44989744 11247436 2345940 94.8 01:58:06 > /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native > -jar /usr/hdp/current/knox-server/bin/ga > > teway.jar > > 24703 44989908 11247477 2515100 100 02:05:13 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990072 11247518 2653252 105 02:12:18 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990244 11247561 2777268 110 02:19:26 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878924 113 02:23:04 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878924 112 02:23:04 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878872 112 02:23:04 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878872 111 02:23:04 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878792 111 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878792 110 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878748 110 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878748 110 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878660 109 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878660 109 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878580 108 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878580 108 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878500 107 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878500 107 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878392 107 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878392 106 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878348 106 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878348 105 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878268 105 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878268 105 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878124 104 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878124 104 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878016 104 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > 24703 44990324 11247581 2878016 103 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java > -Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar > /usr/hdp/current/knox-server/bin/gat > > eway.jar > > > > Has anyone seen this behaviour with knox ? > > > Following is my Knox topology, I am using LDAP authentication > > > > <topology> > > > <gateway> > > > <provider> > > <role>authentication</role> > > <name>ShiroProvider</name> > > <enabled>true</enabled> > > <param> > > <name>sessionTimeout</name> > > <value>30</value> > > </param> > > <param> > > <name>main.ldapRealm</name> > > <value>org.apache.hadoop.gatew > ay.shirorealm.KnoxLdapRealm</value> > > </param> > > <param> > > <name>main.ldapRealm.userDnTemplate</name> > > <value>uid={0},ou=people,dc=ha > doop,dc=apache,dc=org</value> > > </param> > > <param> > > <name>main.ldapRealm.contextFactory.url</name> > > <value>ldap://{{knox_host_name}}:33389</value> > > </param> > > <param> > > <name>main.ldapRealm.contextFa > ctory.authenticationMechanism</name> > > <value>simple</value> > > </param> > > <param> > > <name>urls./**</name> > > <value>authcBasic</value> > > </param> > > </provider> > > > <provider> > > <role>authorization</role> > > <name>AclsAuthz</name> > > <enabled>true</enabled> > > <param> > > <name>knox.acl</name> > > <value>admin;*;*</value> > > </param> > > </provider> > > > <provider> > > <role>identity-assertion</role> > > <name>Default</name> > > <enabled>true</enabled> > > </provider> > > > </gateway> > > > <service> > > <role>KNOX</role> > > </service> > > > </topology> > > > Thanks, > > Vaibhav Kulkarni > >
