Cool, monitoring the transaction makes much sense I think. Interesting to see this tool, I didn't know about it yet. For some reason it replicates most of the logic in FileSnap and FileTxnLog classes, but at the same time it uses ZooKeeper as a dependency.
Regards, Andor On Wed, Mar 7, 2018 at 9:33 AM, Nikhil Bafna <zod...@gmail.com> wrote: > As you mentioned, enabling trace/debug might impact performance. > > > Here's the approach I'm using now. > > - > > Parse the Zookeeper transaction logs. The best repo I've found to do > this is https://github.com/alenca/zklogtool. It's feature packed. The > only downside is that it won't record the read queries. > > > - > zodvik > > On Mon, Mar 5, 2018 at 5:42 PM, Andor Molnar <an...@cloudera.com> wrote: > > > Hi Zod, > > > > Have you checked TRACE logging of client requests? ( > > http://zookeeper.apache.org/doc/r3.4.11/zookeeperAdmin. > html#sc_authOptions > > ) > > Not sure how much different is from what you refer as 'debug logs', but > it > > might worth a try. It generates log messages like: > > > > TRACE [CommitProcWorkThread-1:ZooTrace@88][] - > > :Esessionid:0x8061f60cfd200000 type:getChildren2 cxid:0x6 > > zxid:0xfffffffffffffffe txntype:unknown reqpath:/ > > TRACE [CommitProcWorkThread-1:ZooTrace@88][] - > > :Esessionid:0x8061f60cfd200000 type:getData cxid:0x7 > > zxid:0xfffffffffffffffe txntype:unknown reqpath:/andor2 > > > > Which looks like quite easy to parse and contains all required > information. > > > > Regards, > > Andor > > > > > > On Thu, Mar 1, 2018 at 5:33 AM, Zod <zod...@gmail.com> wrote: > > > > > Unfortunately, parsing debug log is no-go since it doesn't print the > > > complete command. > > > > > > I've 2 approached in mind as of now > > > > > > 1. Modify of org.apache.zookeeper.server.LogFormatter to support > > > "tailing". > > > 2. Use ngrep on my local to monitor the network traffic to remote ZK > > host & > > > port 2181 > > > > > > Will get back with what works better. > > > > > > - > > > zodvik > > > > > > On Thu, Mar 1, 2018 at 1:22 AM, Andor Molnar <an...@cloudera.com> > wrote: > > > > > > > Hi Zod, > > > > > > > > I think there's no such thing in ZooKeeper currently other than debug > > > logs. > > > > You might want to attach some processing engine to the debug log feed > > and > > > > parse and output data which is important for you into MySQL or Redis > > > > database. > > > > > > > > The downside of this approach is debug logging has significant > > > performance > > > > impact on the entire ensemble, but it might be acceptable for you > > > analysis. > > > > > > > > Regards, > > > > Andor > > > > > > > > > > > > > > > > On Tue, Feb 27, 2018 at 11:26 PM, Zod <zod...@gmail.com> wrote: > > > > > > > > > I want see all commands issued to my Zookeeper cluster, something > > like > > > > > general log in MySQL or "monitor" command in Redis. How can this be > > > done? > > > > > > > > > > The intent is to see how Storm uses Zookeeper (for state > management & > > > > > ack-ing) and a good way to would be to run a sample topology and > see > > > all > > > > > the commands issued to Zookeeper by Storm. > > > > > > > > > > I tried enabling debug log for Zookeeper, but that is insufficient > > has > > > a > > > > > lot of noise. For example, issue a create /node prints > > > > > > > > > > 2018-02-27 18:05:34 ZooKeeperMain [DEBUG] Processing create > > > > > - > > > > > zodvik > > > > > > > > > > > > > > >
