[jira] [Commented] (ZOOKEEPER-2251) Add Client side packet response timeout to avoid infinite wait.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063503#comment-15063503 ] nijel commented on ZOOKEEPER-2251: -- hi [~marshad] and [~suda] I observed this when i am doing reliability test for a banking customer Here we test for any network abnormality and packet drop. Here in the scenario packet is sent and wait for ever. Even if the server is not responding due to any reason, this issue can happen so my opinion is to have this time out since many services' high availability solution depends on zookeeper. > Add Client side packet response timeout to avoid infinite wait. > --- > > Key: ZOOKEEPER-2251 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2251 > Project: ZooKeeper > Issue Type: Bug >Reporter: nijel >Assignee: Arshad Mohammad > Attachments: ZOOKEEPER-2251-01.patch, ZOOKEEPER-2251-02.patch, > ZOOKEEPER-2251-03.patch > > > I came across one issue related to Client side packet response timeout In my > cluster many packet drops happened for some time. > One observation is the zookeeper client got hanged. As per the thread dump it > is waiting for the response/ACK for the operation performed (synchronous API > used here). > I am using > zookeeper.serverCnxnFactory=org.apache.zookeeper.server.NIOServerCnxnFactory > Since only few packets missed there is no DISCONNECTED event occurred. > Need add a "response time out" for the operations or packets. > *Comments from [~rakeshr]* > My observation about the problem:- > * Can use tools like 'Wireshark' to simulate the artificial packet loss. > * Assume there is only one packet in the 'outgoingQueue' and unfortunately > the server response packet lost. Now, client will enter into infinite > waiting. > https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/ClientCnxn.java#L1515 > * Probably we can discuss more about this problem and possible solutions(add > packet ACK timeout or another better approach) in the jira. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-1971) Make JMX remote monitoring port configurable
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14996714#comment-14996714 ] nijel commented on ZOOKEEPER-1971: -- Hi [~arshad.mohammad] thanks for the analysis As per my initial analysis, only one jmx port can be configurable, where as when JMX is enabled it is listening to 2 ports will be open for listening Normally in production clusters, listening to a non-configured port is not recommandable. > Make JMX remote monitoring port configurable > > > Key: ZOOKEEPER-1971 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1971 > Project: ZooKeeper > Issue Type: Improvement > Components: server > Environment: All >Reporter: Biju Nair >Assignee: Arshad Mohammad > > This is a follow-up item from ZOOKEEPER-1948. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-2251) Add Client side packet response timeout to avoid infinite wait.
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948352#comment-14948352
]
nijel commented on ZOOKEEPER-2251:
--
thanks [~arshad.mohammad] for the patch
overall looks good
few comments
1. The log message
{code}
LOG.info("{} configured value is {}.", ZOOKEEPER_REQUEST_TIMEOUT,
148 requestTimeoutProp);
{code}
can be written after the value is read. Or in invalid case duplicate logs will
come
2. Change test class and test method name to reflect timeout feature
3. Number format exception logs looks like not a complete sentence.
> Add Client side packet response timeout to avoid infinite wait.
> ---
>
> Key: ZOOKEEPER-2251
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2251
> Project: ZooKeeper
> Issue Type: Bug
>Reporter: nijel
>Assignee: Arshad Mohammad
> Attachments: ZOOKEEPER-2251-01.patch, ZOOKEEPER-2251-02.patch
>
>
> I came across one issue related to Client side packet response timeout In my
> cluster many packet drops happened for some time.
> One observation is the zookeeper client got hanged. As per the thread dump it
> is waiting for the response/ACK for the operation performed (synchronous API
> used here).
> I am using
> zookeeper.serverCnxnFactory=org.apache.zookeeper.server.NIOServerCnxnFactory
> Since only few packets missed there is no DISCONNECTED event occurred.
> Need add a "response time out" for the operations or packets.
> *Comments from [~rakeshr]*
> My observation about the problem:-
> * Can use tools like 'Wireshark' to simulate the artificial packet loss.
> * Assume there is only one packet in the 'outgoingQueue' and unfortunately
> the server response packet lost. Now, client will enter into infinite
> waiting.
> https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/ClientCnxn.java#L1515
> * Probably we can discuss more about this problem and possible solutions(add
> packet ACK timeout or another better approach) in the jira.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-2251) Add Client side packet response timeout to avoid infinite wait.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948362#comment-14948362 ] nijel commented on ZOOKEEPER-2251: -- One more comment can you add this config in document as well ? > Add Client side packet response timeout to avoid infinite wait. > --- > > Key: ZOOKEEPER-2251 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2251 > Project: ZooKeeper > Issue Type: Bug >Reporter: nijel >Assignee: Arshad Mohammad > Attachments: ZOOKEEPER-2251-01.patch, ZOOKEEPER-2251-02.patch > > > I came across one issue related to Client side packet response timeout In my > cluster many packet drops happened for some time. > One observation is the zookeeper client got hanged. As per the thread dump it > is waiting for the response/ACK for the operation performed (synchronous API > used here). > I am using > zookeeper.serverCnxnFactory=org.apache.zookeeper.server.NIOServerCnxnFactory > Since only few packets missed there is no DISCONNECTED event occurred. > Need add a "response time out" for the operations or packets. > *Comments from [~rakeshr]* > My observation about the problem:- > * Can use tools like 'Wireshark' to simulate the artificial packet loss. > * Assume there is only one packet in the 'outgoingQueue' and unfortunately > the server response packet lost. Now, client will enter into infinite > waiting. > https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/ClientCnxn.java#L1515 > * Probably we can discuss more about this problem and possible solutions(add > packet ACK timeout or another better approach) in the jira. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (ZOOKEEPER-1971) Make JMX remote monitoring port configurable
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nijel reassigned ZOOKEEPER-1971: Assignee: Arshad Mohammad (was: nijel) > Make JMX remote monitoring port configurable > > > Key: ZOOKEEPER-1971 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1971 > Project: ZooKeeper > Issue Type: Improvement > Components: server > Environment: All >Reporter: Biju Nair >Assignee: Arshad Mohammad > > This is a follow-up item from ZOOKEEPER-1948. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZOOKEEPER-2287) Audit logging the zookeeper operations
nijel created ZOOKEEPER-2287: Summary: Audit logging the zookeeper operations Key: ZOOKEEPER-2287 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2287 Project: ZooKeeper Issue Type: New Feature Reporter: nijel As of now Zookeeper does not support auditing the user operations This is a very important tracability in distributed cluster to trace the operations We can have a separate logger and log file. Can start with normal node change operations. Please share your thoughts ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-1956) Support Cleanup script in windows
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730286#comment-14730286 ] nijel commented on ZOOKEEPER-1956: -- [~guy.moshkowich] Sorry for late reply Did you delete the patch ? could not find the file ! > Support Cleanup script in windows > - > > Key: ZOOKEEPER-1956 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1956 > Project: ZooKeeper > Issue Type: Bug >Affects Versions: 3.4.6, 3.5.0 >Reporter: nijel >Priority: Minor > > The script zkCleanup.sh support cleaning the zk data in linux system. > The same function needs to be supported in windows also -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-1956) Support Cleanup script in windows
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730285#comment-14730285 ] nijel commented on ZOOKEEPER-1956: -- [~guy.moshkowich] Sorry for late reply :( Did you delete the patch ? > Support Cleanup script in windows > - > > Key: ZOOKEEPER-1956 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1956 > Project: ZooKeeper > Issue Type: Bug >Affects Versions: 3.4.6, 3.5.0 >Reporter: nijel >Priority: Minor > > The script zkCleanup.sh support cleaning the zk data in linux system. > The same function needs to be supported in windows also -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-2251) Add Client side packet response timeout to avoid infinite wait.
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14728996#comment-14728996
]
nijel commented on ZOOKEEPER-2251:
--
hi [~suda]
i got this issue from one of our site. The issue is from ResourceManager.
I could get very few info. The thread which got blocked is as follows
{code}
StandByTransitionThread Handler" daemon prio=10 tid=0x0119a800
nid=0x641e3 in Object.wait() [0x7ff3f67ab000]
1836java.lang.Thread.State: WAITING (on object monitor)
1837 at java.lang.Object.wait(Native Method)
1838 - waiting on <0x0007855f2010> (a
org.apache.zookeeper.ClientCnxn$Packet)
1839 at java.lang.Object.wait(Object.java:503)
1840 at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1414)
1841 - locked <0x0007855f2010> (a
org.apache.zookeeper.ClientCnxn$Packet)
1842 at org.apache.zookeeper.ClientCnxn.close(ClientCnxn.java:1386)
1843 at org.apache.zookeeper.ZooKeeper.close(ZooKeeper.java:677)
1844 - locked <0x00078660daf0> (a org.apache.zookeeper.ZooKeeper)
1845 at
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.closeZkClients(ZKRMStateStore.java:360)
1846 - locked <0x000781087940> (a
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore)
1847 at
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.closeInternal(ZKRMStateStore.java:382)
1848 - locked <0x000781087940> (a
org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore)
1849 at
org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.serviceStop(RMStateStore.java:486)
1850 at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
1851 - locked <0x0007823b2a80> (a java.lang.Object)
1852 at
org.apache.hadoop.service.AbstractService.close(AbstractService.java:250)
1853 at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStop(ResourceManager.java:549)
1854 at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
1855 - locked <0x0007823d0610> (a java.lang.Object)
1856 at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.stopActiveServices(ResourceManager.java:958)
1857 at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToStandby(ResourceManager.java:1018)
1858 - locked <0x000780059c60> (a
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager)
1859 at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.handleTransitionToStandBy(ResourceManager.java:703)
1860 at
org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$StandByTransitionThread.run(RMStateStore.java:884)
{code}
My analysis is that it is waiting for the packet response
Hope this helps
> Add Client side packet response timeout to avoid infinite wait.
> ---
>
> Key: ZOOKEEPER-2251
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2251
> Project: ZooKeeper
> Issue Type: Bug
>Reporter: nijel
>
> I came across one issue related to Client side packet response timeout In my
> cluster many packet drops happened for some time.
> One observation is the zookeeper client got hanged. As per the thread dump it
> is waiting for the response/ACK for the operation performed (synchronous API
> used here).
> I am using
> zookeeper.serverCnxnFactory=org.apache.zookeeper.server.NIOServerCnxnFactory
> Since only few packets missed there is no DISCONNECTED event occurred.
> Need add a "response time out" for the operations or packets.
> *Comments from [~rakeshr]*
> My observation about the problem:-
> * Can use tools like 'Wireshark' to simulate the artificial packet loss.
> * Assume there is only one packet in the 'outgoingQueue' and unfortunately
> the server response packet lost. Now, client will enter into infinite
> waiting.
> https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/ClientCnxn.java#L1515
> * Probably we can discuss more about this problem and possible solutions(add
> packet ACK timeout or another better approach) in the jira.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-2238) Support limiting the maximum number of connections/clients to a zookeeper server.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14728513#comment-14728513 ] nijel commented on ZOOKEEPER-2238: -- [~eribeiro] Thanks for taking up the issue. Please let me know whether you started the work. > Support limiting the maximum number of connections/clients to a zookeeper > server. > - > > Key: ZOOKEEPER-2238 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2238 > Project: ZooKeeper > Issue Type: Improvement >Reporter: nijel >Assignee: Edward Ribeiro > > Currently zookeeper have the feature of limiting the maximum number of > connection/client per IP or Host (maxClientCnxns). > But to safe guard zookeeper server from DoS attack due to many clients from > different IPs, it is better to have a limit of total number of > connections/clients to a a single member of the ZooKeeper ensemble as well. > So the idea is to introduce a new configuration to limit the maximum number > of total connections/clients. > Please share your thoughts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZOOKEEPER-2251) Add Client side packet response timeout to avoid infinite wait.
nijel created ZOOKEEPER-2251: Summary: Add Client side packet response timeout to avoid infinite wait. Key: ZOOKEEPER-2251 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2251 Project: ZooKeeper Issue Type: Bug Reporter: nijel I came across one issue related to Client side packet response timeout In my cluster many packet drops happened for some time. One observation is the zookeeper client got hanged. As per the thread dump it is waiting for the response/ACK for the operation performed (synchronous API used here). I am using zookeeper.serverCnxnFactory=org.apache.zookeeper.server.NIOServerCnxnFactory Since only few packets missed there is no DISCONNECTED event occurred. Need add a response time out for the operations or packets. *Comments from [~rakeshr]* My observation about the problem:- * Can use tools like 'Wireshark' to simulate the artificial packet loss. * Assume there is only one packet in the 'outgoingQueue' and unfortunately the server response packet lost. Now, client will enter into infinite waiting. https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/ClientCnxn.java#L1515 * Probably we can discuss more about this problem and possible solutions(add packet ACK timeout or another better approach) in the jira. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZOOKEEPER-2238) Support limiting the maximum number of connections/clients to a zookeeper server.
nijel created ZOOKEEPER-2238: Summary: Support limiting the maximum number of connections/clients to a zookeeper server. Key: ZOOKEEPER-2238 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2238 Project: ZooKeeper Issue Type: Improvement Reporter: nijel Currently zookeeper have the feature of limiting the maximum number of connection/client per IP or Host (maxClientCnxns). But to safe guard zookeeper server from DoS attack due to many clients from different IPs, it is better to have a limit of total number of connections/clients to a a single member of the ZooKeeper ensemble as well. So the idea is to introduce a new configuration to limit the maximum number of total connections/clients. Please share your thoughts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-2215) Four letter command don't have kerberos authentication
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14584454#comment-14584454 ] nijel commented on ZOOKEEPER-2215: -- Thanks [~surendrasingh] for reporting I also came across this point Since it is directly reading from socket i think it may not be possible to add SASL auth since the client cant pass the credentials One option is to add IP/HOST filter to make it secure bit :) What is your opinion ? Four letter command don't have kerberos authentication --- Key: ZOOKEEPER-2215 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2215 Project: ZooKeeper Issue Type: Bug Reporter: surendra singh lilhore echo dump | netcat IP port -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-1983) Append to zookeeper.out (not overwrite) to support logrotation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504714#comment-14504714 ] nijel commented on ZOOKEEPER-1983: -- hi Can we think of out file rotation ? say we will keep last 5 or 10 out files. The rolling will happen on each restart. Append to zookeeper.out (not overwrite) to support logrotation -- Key: ZOOKEEPER-1983 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1983 Project: ZooKeeper Issue Type: Bug Components: server Affects Versions: 3.3.5, 3.3.6, 3.4.6 Environment: CentOS 5.x (and probably any Linux distribution for that matter) Reporter: Shyamal Prasad Assignee: Shyamal Prasad Fix For: 3.5.2, 3.6.0 Attachments: ZK1983.patch, ZOOKEEPER-1983.patch Currently zkServer.sh will redirect output to zookeeper.out using a simple shell redirect. When logrotate (and similar tools) are used to rotate the zookeeper.out file with the 'copytruncate' semantics (copy the file, truncate it to zero bytes) the next write results in a sparse file with the write at the offset of the last file. Effectively the log file is now full a null bytes and it is hard to read/use the file (and the rotated copies). Even worse, the result is zookeeper.out file only gets larger (though sparse) and after a while on a chatty system it takes significant CPU resources to compress the file (which is all nulls!) The simple fix is to append to the file () instead of a simple redirection () This issue was found in a 3.3.5 production system, however code in trunk has the same issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (ZOOKEEPER-1971) Make JMX remote monitoring port configurable
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nijel reassigned ZOOKEEPER-1971: Assignee: nijel Make JMX remote monitoring port configurable Key: ZOOKEEPER-1971 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1971 Project: ZooKeeper Issue Type: Improvement Components: server Environment: All Reporter: Biju Nair Assignee: nijel This is a follow-up item from ZOOKEEPER-1948. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (ZOOKEEPER-1971) Make JMX remote monitoring port configurable
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nijel reassigned ZOOKEEPER-1971: Assignee: nijel Make JMX remote monitoring port configurable Key: ZOOKEEPER-1971 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1971 Project: ZooKeeper Issue Type: Improvement Components: server Environment: All Reporter: Biju Nair Assignee: nijel This is a follow-up item from ZOOKEEPER-1948. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (ZOOKEEPER-1971) Make JMX remote monitoring port configurable
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074227#comment-14074227 ] nijel commented on ZOOKEEPER-1971: -- can i take up this ? Make JMX remote monitoring port configurable Key: ZOOKEEPER-1971 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1971 Project: ZooKeeper Issue Type: Improvement Components: server Environment: All Reporter: Biju Nair This is a follow-up item from ZOOKEEPER-1948. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (ZOOKEEPER-1971) Make JMX remote monitoring port configurable
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nijel reassigned ZOOKEEPER-1971: Assignee: (was: nijel) wrongly assigned Make JMX remote monitoring port configurable Key: ZOOKEEPER-1971 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1971 Project: ZooKeeper Issue Type: Improvement Components: server Environment: All Reporter: Biju Nair This is a follow-up item from ZOOKEEPER-1948. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (ZOOKEEPER-1959) Move zkEnv files to conf folder, since it is expected to be configured by user
nijel created ZOOKEEPER-1959: Summary: Move zkEnv files to conf folder, since it is expected to be configured by user Key: ZOOKEEPER-1959 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1959 Project: ZooKeeper Issue Type: Improvement Reporter: nijel Move zkEnv files to conf folder This is the common pattern followed across other hadoop components -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (ZOOKEEPER-1960) Improve logs in PurgeTxnLog.java to give more details
nijel created ZOOKEEPER-1960: Summary: Improve logs in PurgeTxnLog.java to give more details Key: ZOOKEEPER-1960 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1960 Project: ZooKeeper Issue Type: Improvement Reporter: nijel Improve logs in PurgeTxnLog.java to give more details Suggest to add logs in following scenarios 1. If no file to purge (deletion list is empty) - INFO 2. Add logs for debug purpose (txnLog, dataDir and snapDir, passed arguements) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (ZOOKEEPER-1952) Default log directory and file name can be changed
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nijel updated ZOOKEEPER-1952: - Attachment: ZOOKEEPER-1952-1.patch Attaching patch Default log directory and file name can be changed -- Key: ZOOKEEPER-1952 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1952 Project: ZooKeeper Issue Type: Bug Affects Versions: 3.4.6 Reporter: nijel Assignee: nijel Priority: Minor Attachments: ZOOKEEPER-1952-1.patch The log folder and log file name is configurable now. The default log folder is . in distribution. So the log file (zookeeper.out) will be placed in bin folder Can this be changed to zk_home/logs/zookeeperserver-hostname.log ? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (ZOOKEEPER-1952) Default log directory and file name can be changed
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nijel updated ZOOKEEPER-1952: - Attachment: (was: ZOOKEEPER-1952-1.patch) Default log directory and file name can be changed -- Key: ZOOKEEPER-1952 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1952 Project: ZooKeeper Issue Type: Bug Affects Versions: 3.4.6 Reporter: nijel Assignee: nijel Priority: Minor The log folder and log file name is configurable now. The default log folder is . in distribution. So the log file (zookeeper.out) will be placed in bin folder Can this be changed to zk_home/logs/zookeeperserver-hostname.log ? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (ZOOKEEPER-1952) Default log directory and file name can be changed
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053447#comment-14053447 ] nijel commented on ZOOKEEPER-1952: -- Thanks Flavio for taking a look in the issue Attached patch with the changes. Had to change zkCli and server scripts for adding the log file name since there is no unified script. Changed the default logger to Rolling file. Default log directory and file name can be changed -- Key: ZOOKEEPER-1952 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1952 Project: ZooKeeper Issue Type: Bug Affects Versions: 3.4.6 Reporter: nijel Assignee: nijel Priority: Minor Attachments: ZOOKEEPER-1952-1.patch The log folder and log file name is configurable now. The default log folder is . in distribution. So the log file (zookeeper.out) will be placed in bin folder Can this be changed to zk_home/logs/zookeeperserver-hostname.log ? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (ZOOKEEPER-1952) Default log directory and file name can be changed
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nijel updated ZOOKEEPER-1952: - Attachment: ZOOKEEPER-1952-1.patch Updated the patch Default log directory and file name can be changed -- Key: ZOOKEEPER-1952 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1952 Project: ZooKeeper Issue Type: Bug Affects Versions: 3.4.6 Reporter: nijel Assignee: nijel Priority: Minor Attachments: ZOOKEEPER-1952-1.patch The log folder and log file name is configurable now. The default log folder is . in distribution. So the log file (zookeeper.out) will be placed in bin folder Can this be changed to zk_home/logs/zookeeperserver-hostname.log ? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (ZOOKEEPER-1948) Enable JMX remote monitoring
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053479#comment-14053479 ] nijel commented on ZOOKEEPER-1948: -- bq.Is this related to ZOOKEEPER-1346? From the discussion i understand the issue ZOOKEEPER-1346 is for supporting a new REST interface for 4wls. Even after this implementation users will have option to use the remote JMX monitoring by enabling JMX server. But if the remote monitoring is enabled, then it will use 2 extra ports, one of which is random. So my suggestion was to implement RMI based custom JMX server where the ports can be configured. BTW this issue (Enable JMX remote monitoring) is because of missing arguement *com.sun.management.jmxremote.authenticate*=false As per code remote monitoring should be enabled since default value of JMXLOCALONLY=false. I added this property as -D args and it worked. i think need to add this in script Enable JMX remote monitoring Key: ZOOKEEPER-1948 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1948 Project: ZooKeeper Issue Type: Improvement Components: server Environment: All Reporter: Biju Nair The zooker server start up script includes the option to enable jmx monitoring but only locally. Can we update the script so that remote monitoring can also be enabled which will help in data collection and monitoring through a centralized monitoring tool. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (ZOOKEEPER-1122) start and stop commands are not present in zkServer.cmd
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053515#comment-14053515 ] nijel commented on ZOOKEEPER-1122: -- hi Alexander and Iurii Are you working on this issue ? start and stop commands are not present in zkServer.cmd --- Key: ZOOKEEPER-1122 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1122 Project: ZooKeeper Issue Type: Improvement Components: scripts Affects Versions: 3.3.3 Environment: Windows Reporter: Alexander Osadchiy Labels: patch Fix For: 3.6.0 Attachments: zkServer.cmd, zkServer.cmd Now ZooKeeper server can be started and stoped from Unix-based systems using script bin/zkServer.sh: bin/zkServer.sh start - to start server; bin/zkServer.sh stop - to stop server. There are no start and stop commands in script zkServer.cmd (for Windows). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (ZOOKEEPER-1956) Support Cleanup script in windows
nijel created ZOOKEEPER-1956: Summary: Support Cleanup script in windows Key: ZOOKEEPER-1956 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1956 Project: ZooKeeper Issue Type: Bug Reporter: nijel Priority: Minor The script zkCleanup.sh support cleaning the zk data in linux system. The same function needs to be supported in windows also -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (ZOOKEEPER-545) investigate use of realtime gc as the recommened default for server vm
[ https://issues.apache.org/jira/browse/ZOOKEEPER-545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053527#comment-14053527 ] nijel commented on ZOOKEEPER-545: - Users will face challenge in setting other GC options also (like mewSize, mx, gcInterval, ms). Can we have a profile based GC options ? We can ship a *gc-opts* file with multiple profiles (low,medium and high). Each profile will have set of GC options. The profiles are named in the order of memory configured. Like high can have mx as 4 gb and related configurations. Low can have 512 as mx and related configurations. So user can just configure the profile name. As we are getting more improvements, comments can update these values in the file. investigate use of realtime gc as the recommened default for server vm -- Key: ZOOKEEPER-545 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-545 Project: ZooKeeper Issue Type: Improvement Components: server Reporter: Patrick Hunt Priority: Critical Labels: newbie Fix For: 3.6.0 We currently don't recommend that ppl use the realtime gc when running the server, we probably should. Before we do so we need to verify that it works. We should make it the default for all our tests. concurrent vs g2 or whatever it's called (new in 1.6_15 or something?) Update all scripts to specify this option update documentation to include this option and add section in the dev/ops docs detailing it's benefits (in particular latency effects of gc) Also, -server option? any benefit for us to recommend this as well? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (ZOOKEEPER-1948) Enable JMX remote monitoring
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049855#comment-14049855 ] nijel commented on ZOOKEEPER-1948: -- Adding to the suggestion. If the remote JMX server is enabled, it will use one Random port. This may be in convenient for cluster managers. So i suggest we can have a custom JMX server which is based on RMI Refer : https://issues.apache.org/jira/browse/HBASE-10289 Enable JMX remote monitoring Key: ZOOKEEPER-1948 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1948 Project: ZooKeeper Issue Type: Improvement Components: server Environment: All Reporter: Biju Nair The zooker server start up script includes the option to enable jmx monitoring but only locally. Can we update the script so that remote monitoring can also be enabled which will help in data collection and monitoring through a centralized monitoring tool. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (ZOOKEEPER-1952) Default log directory and file name can be changed
nijel created ZOOKEEPER-1952: Summary: Default log directory and file name can be changed Key: ZOOKEEPER-1952 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1952 Project: ZooKeeper Issue Type: Bug Affects Versions: 3.4.6 Reporter: nijel Priority: Minor The log folder and log file name is configurable now. The default log folder is . in distribution. So the log file (zookeeper.out) will be placed in bin folder Can this be changed to zk_home/logs/zookeeperserver-hostname.log ? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (BOOKKEEPER-240) Need a deadlock checking mechanism to avoid deadlock
[ https://issues.apache.org/jira/browse/BOOKKEEPER-240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nijel updated BOOKKEEPER-240: - Attachment: JCarder Doc.pdf Details of Running Jcarder. Need a deadlock checking mechanism to avoid deadlock Key: BOOKKEEPER-240 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-240 Project: Bookkeeper Issue Type: Improvement Reporter: Sijie Guo Attachments: JCarder Doc.pdf per discussion on BOOKKEEPER-215, we need a deadlock checking mechanism. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-296) It's better provide stop script for bookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nijel updated BOOKKEEPER-296: - Attachment: BOOKKEEPER-296.4.patch It's better provide stop script for bookie -- Key: BOOKKEEPER-296 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-296 Project: Bookkeeper Issue Type: Bug Components: bookkeeper-server Affects Versions: 4.0.0 Reporter: Brahma Reddy Battula Attachments: BOOKKEEPER-296.1.patch, BOOKKEEPER-296.2.patch, BOOKKEEPER-296.3.patch, BOOKKEEPER-296.4.patch, BOOKKEEPER-296.patch Currently there is no command to stop bookie in 4.0.0..It's better to provide stop command for bookie. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-296) It's better provide stop script for bookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404487#comment-13404487 ] nijel commented on BOOKKEEPER-296: -- Thanks for the comments one question about the dailyrolling appender. this appender doesn't have maxFileSize property as below Commented the particular property btw, it is better to put BOOKIE_STOP_TIMEOUT BOOKIE_PID_DIR in bkenv.sh as default and comment out them Added in bkenv.sh Updated the new patch It's better provide stop script for bookie -- Key: BOOKKEEPER-296 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-296 Project: Bookkeeper Issue Type: Bug Components: bookkeeper-server Affects Versions: 4.0.0 Reporter: Brahma Reddy Battula Attachments: BOOKKEEPER-296.1.patch, BOOKKEEPER-296.2.patch, BOOKKEEPER-296.3.patch, BOOKKEEPER-296.4.patch, BOOKKEEPER-296.patch Currently there is no command to stop bookie in 4.0.0..It's better to provide stop command for bookie. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-296) It's better provide stop script for bookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13398555#comment-13398555 ] nijel commented on BOOKKEEPER-296: -- Thanks Uma for reviewing Updated the comments. Pls review the updated patch It's better provide stop script for bookie -- Key: BOOKKEEPER-296 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-296 Project: Bookkeeper Issue Type: Bug Components: bookkeeper-server Affects Versions: 4.0.0 Reporter: Brahma Reddy Battula Attachments: BOOKKEEPER-296.1.patch, BOOKKEEPER-296.2.patch, BOOKKEEPER-296.3.patch, BOOKKEEPER-296.patch Currently there is no command to stop bookie in 4.0.0..It's better to provide stop command for bookie. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-296) It's better provide stop script for bookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nijel updated BOOKKEEPER-296: - Attachment: BOOKKEEPER-296.2.patch Updated patch It's better provide stop script for bookie -- Key: BOOKKEEPER-296 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-296 Project: Bookkeeper Issue Type: Bug Components: bookkeeper-server Affects Versions: 4.0.0 Reporter: Brahma Reddy Battula Attachments: BOOKKEEPER-296.1.patch, BOOKKEEPER-296.2.patch, BOOKKEEPER-296.patch Currently there is no command to stop bookie in 4.0.0..It's better to provide stop command for bookie. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-296) It's better provide stop script for bookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nijel updated BOOKKEEPER-296: - Attachment: BOOKKEEPER-296.1.patch Changed the echo statements It's better provide stop script for bookie -- Key: BOOKKEEPER-296 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-296 Project: Bookkeeper Issue Type: Bug Components: bookkeeper-server Affects Versions: 4.0.0 Reporter: Brahma Reddy Battula Attachments: BOOKKEEPER-296.1.patch, BOOKKEEPER-296.patch Currently there is no command to stop bookie in 4.0.0..It's better to provide stop command for bookie. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-296) It's better provide stop script for bookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nijel updated BOOKKEEPER-296: - Attachment: BOOKKEEPER-296.patch It's better provide stop script for bookie -- Key: BOOKKEEPER-296 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-296 Project: Bookkeeper Issue Type: Bug Components: bookkeeper-server Affects Versions: 4.0.0 Reporter: Brahma Reddy Battula Attachments: BOOKKEEPER-296.patch Currently there is no command to stop bookie in 4.0.0..It's better to provide stop command for bookie. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-296) It's better provide stop script for bookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13396014#comment-13396014 ] nijel commented on BOOKKEEPER-296: -- The following modificatiosn are done to make the script better 1. Implemented the pid based stop command 2. While stopping used kill command to stop it gracefully and after some timeout (default 3 sec) used kill -9 to stop the bookie 3. While starting the pid file is being checked, if available the process existance is checked and if not continue with startup. The same is implemented for localbookie Attached the patch for thie work. Pls review It's better provide stop script for bookie -- Key: BOOKKEEPER-296 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-296 Project: Bookkeeper Issue Type: Bug Components: bookkeeper-server Affects Versions: 4.0.0 Reporter: Brahma Reddy Battula Attachments: BOOKKEEPER-296.patch Currently there is no command to stop bookie in 4.0.0..It's better to provide stop command for bookie. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
