Hi all, I have a long interest in pub-sub and have written several variants of proxies/bridges to JMS ( ActiveMQ, HornetQ and some others) in Java and I share the motivations behind Kafka project (esp. the idea of keeping the consumption state at the consumer side really speaks to my heart and also the focus on persistence).
So I started writing some proofs of concept for the 50GB of daily event logs I work with and which I would like to turn into a real time streams rather than offline hive processes. I often use php for proof-of-concepts because it's easy to write and throw away but after a few hours of using and looking at the Kafka PHP Client which is in the incubator trunk I have ended up writing and alternative because the existing client does few things which don't work for me ( mainly the fact that it loads the entire fetch-response into memory and then processes internal buffer plus few other things.. ) and doesn't do few things I need ( mainly doesn't try to deal with the long offsets and decompression ) Please have a look if you're interested: http://svn.xp-dev.com/svn/kafkaphp/trunk/ (Btw, having a robust and simple php api would be an easy step towards a restful proxy which could be later turned into scala/java underneath once the restful api would get stable) Michal
