Re: [PATCH v7 6/7] ieot: add Index Entry Offset Table (IEOT) extension

2018-10-02 Thread Duy Nguyen
On Tue, Oct 2, 2018 at 6:34 PM Ben Peart wrote: > >> + offset = lseek(newfd, 0, SEEK_CUR) + > >> write_buffer_len; > > > > This only works correctly if the ce_write_entry() from the last > > iteration has flushed everything to out to newfd. Maybe it does, but > > it's error

Re: [PATCH v7 6/7] ieot: add Index Entry Offset Table (IEOT) extension

2018-10-02 Thread Ben Peart
On 10/1/2018 12:27 PM, Duy Nguyen wrote: On Mon, Oct 1, 2018 at 3:46 PM Ben Peart wrote: @@ -1888,6 +1890,23 @@ static size_t estimate_cache_size(size_t ondisk_size, unsigned int entries) return ondisk_size + entries * per_entry; } +struct index_entry_offset +{ + /* starti

Re: [PATCH v7 6/7] ieot: add Index Entry Offset Table (IEOT) extension

2018-10-01 Thread Duy Nguyen
On Mon, Oct 1, 2018 at 3:46 PM Ben Peart wrote: > @@ -1888,6 +1890,23 @@ static size_t estimate_cache_size(size_t ondisk_size, > unsigned int entries) > return ondisk_size + entries * per_entry; > } > > +struct index_entry_offset > +{ > + /* starting byte offset into index file, co

[PATCH v7 6/7] ieot: add Index Entry Offset Table (IEOT) extension

2018-10-01 Thread Ben Peart
From: Ben Peart This patch enables addressing the CPU cost of loading the index by adding additional data to the index that will allow us to efficiently multi- thread the loading and conversion of cache entries. It accomplishes this by adding an (optional) index extension that is a table of offs