As you mentioned, enabling trace/debug might impact performance.
Here's the approach I'm using now. - Parse the Zookeeper transaction logs. The best repo I've found to do this is https://github.com/alenca/zklogtool. It's feature packed. The only downside is that it won't record the read queries. - zodvik On Mon, Mar 5, 2018 at 5:42 PM, Andor Molnar <[email protected]> wrote: > Hi Zod, > > Have you checked TRACE logging of client requests? ( > http://zookeeper.apache.org/doc/r3.4.11/zookeeperAdmin.html#sc_authOptions > ) > Not sure how much different is from what you refer as 'debug logs', but it > might worth a try. It generates log messages like: > > TRACE [CommitProcWorkThread-1:ZooTrace@88][] - > :Esessionid:0x8061f60cfd200000 type:getChildren2 cxid:0x6 > zxid:0xfffffffffffffffe txntype:unknown reqpath:/ > TRACE [CommitProcWorkThread-1:ZooTrace@88][] - > :Esessionid:0x8061f60cfd200000 type:getData cxid:0x7 > zxid:0xfffffffffffffffe txntype:unknown reqpath:/andor2 > > Which looks like quite easy to parse and contains all required information. > > Regards, > Andor > > > On Thu, Mar 1, 2018 at 5:33 AM, Zod <[email protected]> wrote: > > > Unfortunately, parsing debug log is no-go since it doesn't print the > > complete command. > > > > I've 2 approached in mind as of now > > > > 1. Modify of org.apache.zookeeper.server.LogFormatter to support > > "tailing". > > 2. Use ngrep on my local to monitor the network traffic to remote ZK > host & > > port 2181 > > > > Will get back with what works better. > > > > - > > zodvik > > > > On Thu, Mar 1, 2018 at 1:22 AM, Andor Molnar <[email protected]> wrote: > > > > > Hi Zod, > > > > > > I think there's no such thing in ZooKeeper currently other than debug > > logs. > > > You might want to attach some processing engine to the debug log feed > and > > > parse and output data which is important for you into MySQL or Redis > > > database. > > > > > > The downside of this approach is debug logging has significant > > performance > > > impact on the entire ensemble, but it might be acceptable for you > > analysis. > > > > > > Regards, > > > Andor > > > > > > > > > > > > On Tue, Feb 27, 2018 at 11:26 PM, Zod <[email protected]> wrote: > > > > > > > I want see all commands issued to my Zookeeper cluster, something > like > > > > general log in MySQL or "monitor" command in Redis. How can this be > > done? > > > > > > > > The intent is to see how Storm uses Zookeeper (for state management & > > > > ack-ing) and a good way to would be to run a sample topology and see > > all > > > > the commands issued to Zookeeper by Storm. > > > > > > > > I tried enabling debug log for Zookeeper, but that is insufficient > has > > a > > > > lot of noise. For example, issue a create /node prints > > > > > > > > 2018-02-27 18:05:34 ZooKeeperMain [DEBUG] Processing create > > > > - > > > > zodvik > > > > > > > > > >
