Re: Hackathon notes 3/21/2011

2011-03-26 Thread Ted Dunning
I was thinking in terms of a more general test where read-modify-write operations were being used. It is also helpful to have some tests of simple over-write. If there is a percentage of ops that are reads and if data can be determined to be prima facie valid or not, then this can be done during

Re: Hackathon notes 3/21/2011

2011-03-26 Thread Todd Lipcon
On Sat, Mar 26, 2011 at 3:53 PM, Ted Dunning wrote: > Hmm... > > Yeah. I hear that "scrapping YCSB" meme a lot. > > Do you not worry about verifying intermediate results when over-writing > data? > Not sure what you mean by this? The design of this system test is basically to create virtual "l

Re: Hackathon notes 3/21/2011

2011-03-26 Thread Ted Dunning
Hmm... Yeah. I hear that "scrapping YCSB" meme a lot. Do you not worry about verifying intermediate results when over-writing data? On Sat, Mar 26, 2011 at 8:51 AM, Todd Lipcon wrote: > Hi Ted, > > I actually ended up scrapping the YCSB approach and built a > system/durability test instead. I

Re: Hackathon notes 3/21/2011

2011-03-26 Thread Todd Lipcon
Hi Ted, I actually ended up scrapping the YCSB approach and built a system/durability test instead. It's an MR job that writes a particular pattern of edits, and a second one that verifies them. I'm in the process of hooking this into our continuous integration system, and will attempt to open sou

Re: Hackathon notes 3/21/2011

2011-03-26 Thread Ted Dunning
Todd, I see ycsb on your list. Where did that go? We have been beating on it as well and have pretty much decided that it is worthless as it stands. My thought is that we need a multi-node version that takes directions about what load to generate via ZK. That is better than a map-reduce based

Hackathon notes 3/21/2011

2011-03-25 Thread Todd Lipcon
Dear HBase developers, Last Monday, several HBase contributors met up at the StumbleUpon offices for a bit of a hackathon. We spent the beginning of the day discussing a few general topics, and then from about 11am through 7pm or so most of us hunkered down to hacking on various projects. I was th