All, Synopsis: I can't get cache tiering to work in HAMMER on RHEL7.
Process: 1. Fresh install of HAMMER on RHEL7 went well. 2. Crush map adapted to provide two "root" level resources a. "ctstorage", to use as a cache tier based on very high-performance, high IOPS SSD (intrinsic journal). 2 OSDs. b. "ecstorage", to use as an erasure-coded poolbased on low-performance, cost effective storage (extrinsic journal). 12 OSDs. 3. Established a pool "ctpool", 32 PGs on ctstorage (pool size = min_size = 1). Ran a quick RADOS bench test, all worked as expected. Cleaned up. 4. Established a pool "ecpool", 256 PGs on ecstorage (5+3 profile). Ran a quick RADOS bench test, all worked as expected. Cleaned up. 5. Ensured that both pools were empty (i.e., "rados ls" shows no objects) 6. Put the cache tier on the erasure coded storage (one Bloom hit set, interval 900 seconds), set up the overlay. Used defaults for flushing and eviction. No errors. 7. Started a 3600-second write test to ecpool. Objects piled up in ctpool (as expected) - went past the 40% mark (as expected), then past the 80% mark (unexpected), then ran into the wall (95% full - very unexpected). Using "rados df", I can see that the cache tier is full (duh!) but not one single object lives in the ecpool. Nothing was ever flushed, nothing was ever evicted. Thought I might be misreading that, so I went back to SAR data that I captured during the test: the SSDs were the only [ceph] devices that sustained I/O. I based this experiment on another (much more successful) experiment that I performed using GIANT (.1) on RHEL7 a couple of weeks ago (wherein I used RAM as a cache tier); that all worked. It seems there are at least three possibilities... * I forgot a critical step this time around. * The steps needed for a cache tier in HAMMER are different than the steps needed in GIANT (and different than the documentation online). * There is a problem with HAMMER in the area of cache tier. Has anyone successfully deployed cache-tiering in HAMMER? Did you have to do anything unusual? Do you see any steps that I missed? Regards, -don- ---------------------------------------------------------------------- The information contained in this transmission may be confidential. Any disclosure, copying, or further distribution of confidential information is not permitted unless such privilege is explicitly granted in writing by Quantum. Quantum reserves the right to have electronic communications, including email and attachments, sent across its networks filtered through anti virus and spam software programs and retain such messages in order to comply with applicable data security and retention requirements. Quantum is not responsible for the proper and complete transmission of the substance of this communication or for any delay in its receipt.
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
