Re: Monitoring bytes read per cf

[Peter Schuller]

 It seems to work fine but maybe someone here knows that this is not a good 
 idea 

It depends. If you're trying to figure out what generates actual iops
to disk, it won't really tell you since you're counting all I/O
including that which is cached. Since whatever goes down to disk tends
to be, due to the LRU:ishness of caches, the less frequently accessed
data that tends to make it difficult to judge by numbers that include
all I/O.

-- 
/ Peter Schuller

Re: Monitoring bytes read per cf

[Daniel Doubleday]

Thanks - yes I agree. Didn't want to judge solely based on this figure. 

It should just add to the picture. But since we know access patterns and other 
stats like key and row cache hit ratios we hope to be able to make a more 
educated guess whats going on.

On May 13, 2011, at 9:08 AM, Peter Schuller wrote:

 It seems to work fine but maybe someone here knows that this is not a good 
 idea 
 
 It depends. If you're trying to figure out what generates actual iops
 to disk, it won't really tell you since you're counting all I/O
 including that which is cached. Since whatever goes down to disk tends
 to be, due to the LRU:ishness of caches, the less frequently accessed
 data that tends to make it difficult to judge by numbers that include
 all I/O.
 
 -- 
 / Peter Schuller

Monitoring bytes read per cf

[Daniel Doubleday]

Hi all

got a question for folks with some code insight again.

To be able to better understand where our IO load is coming from we want to 
monitor the number of bytes read from disc per cf. (we love stats)

What I have done is wrapping the FileDataInput in SSTableReader to sum the 
bytes read in CFS. This will only record data file access but that would be 
good enough for us.

It seems to work fine but maybe someone here knows that this is not a good idea 


Cheers,
Daniel

Some code:

SSTableReader:
private static final boolean KEEP_IO_STATISICS = 
Boolean.getBoolean(cassandra.keepIOStats);
public FileDataInput getFileDataInput(DecoratedKey decoratedKey, int 
bufferSize)
{
long position = getPosition(decoratedKey, Operator.EQ);
if (position  0)
return null;

FileDataInput segment = dfile.getSegment(position, bufferSize);
return (KEEP_IO_STATISICS) ? new MonitoringFileDataIInput(metadata, 
segment) : segment; 
}

with MonitoringFileDataIInput

public class MonitoringFileDataIInput implements FileDataInput, Closeable
{

private final FileDataInput fileDataInput;
private final ColumnFamilyStore columnFamilyStore;

public MonitoringFileDataIInput(CFMetaData cfMetaData, FileDataInput 
fileDataInput)
{
columnFamilyStore = 
Table.open(cfMetaData.tableName).getColumnFamilyStore(cfMetaData.cfId);
this.fileDataInput = fileDataInput;
}

@Override
public boolean readBoolean() throws IOException
{
columnFamilyStore.addBytesRead(1);
return fileDataInput.readBoolean();
}

// ... etc

and ColumnFamilyStore
private final AtomicLong bytesRead = new AtomicLong(0L);

@Override // ColumnFamilyStoreMBean
public long getBytesRead()
{
return bytesRead.get();
}

public void addBytesRead(int num) 
{
bytesRead.addAndGet(num);
}