Hi All,
We've been fixing a number of objectcacher bugs to handle races between
slow osd commit replies and various other operations like truncate. I
ran into another problem earlier today with a race between an object
getting evicted from the lru cache (via readx -> trim) and the osd
commit reply. The assertion trace is below.
We've avoided keeping a reference to the object during the commit, but
that means that the object isn't pinned in the lru, and so can come up
for eviction. When it gets evicted, we close the object and hit the
assertion, which we can't do - because we need the object to finish the
commit.
I've pushed a change that needs review in the wip-3431 branch. It
allows the the object to be evicted from the lru cache, but checks that
it can be closed (as we do elsewhere) - and if not, lets the commit
handle the close (via flush...release).
The assertion we hit is:
2012-11-19 09:06:35.187910 7ff143e2f780 1 osdc/ObjectCacher.cc: In
function 'void ObjectCacher::close_object(ObjectCacher::Object*)' thread
7ff143e2f780 time 2012-11-19 09:06:35.186379
osdc/ObjectCacher.cc: 577: FAILED assert(ob>can_close())
ceph version 0.54-641-g4c69f86 (4c69f865ca79328c62635ae32c91bd32b3985613)
1: (ObjectCacher::close_object(ObjectCacher::Object*)+0x135) [0x5c78d5]
2: (ObjectCacher::trim(long, long)+0x820) [0x5c94d0]
3: (ObjectCacher::_readx(ObjectCacher::OSDRead*,
ObjectCacher::ObjectSet*, Context*, bool)+0x21ad) [0x5d92dd]
4: (Client::_read_async(Fh*, unsigned long, unsigned long,
ceph::buffer::list*)+0x3e9) [0x486c09]
5: (Client::_read(Fh*, long, unsigned long,
ceph::buffer::list*)+0x265) [0x49bd65]
6: (Client::ll_read(Fh*, long, long, ceph::buffer::list*)+0x97) [0x49be87]
7: /tmp/cephtest/binary/usr/local/bin/ceph-fuse() [0x4733cf]
8: (()+0x12d5e) [0x7ff1439fdd5e]
9: (fuse_session_loop()+0x75) [0x7ff1439fbd65]
10: (ceph_fuse_ll_main(Client*, int, char const**, int)+0x225) [0x474245]
11: (main()+0x42f) [0x4716ef]
12: (__libc_start_main()+0xed) [0x7ff141ebd76d]
13: /tmp/cephtest/binary/usr/local/bin/ceph-fuse() [0x472e95]
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
