Re: [ceph-users] fibre channel as ceph storage interconnect

On Apr 21, 2016, at 11:10 PM, Schlacta, Christ mailto:aarc...@aarcane.org>> wrote: Would it be worth while development effort to establish a block protocol between the nodes so that something like fibre channel could be used to communicate internally? With 25/100 Ethernet & IB becoming available

Re: [ceph-users] Intel S3710 400GB and Samsung PM863 480GB fio results

As the single job spec being referenced (from Sebastien Han’s blog, as I understand) includes the use of the —sync flag, the HBA and storage bus are unlikely to dominate the test results.

Re: [ceph-users] Configure Ceph client network

Yes: make sure there is an entry in your ceph.conf to align the public (client) network with the IP space of the NIC where you want the ceph IO... [global] public_network = / There is also an option to iso

Re: [ceph-users] Building a Pb EC cluster for a cheaper cold storage

rs mailing list ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Paul Evans Principal Architect Daystrom Technology Group m: 707-479-1034o: 800-656-3224 x511 f: 650-472-4005e: paul.ev...@daystrom.co

Re: [ceph-users] high density machines

power and cooling. That said: the cost savings would need to be solid to choose the FatTwin over 1U boxes, especially as (personally) I dislike lots of front-side cabling in the rack. -- Paul Evans On Sep 3, 2015, at 7:01 AM, Gurvinder Singh mailto:gurvindersinghdah...@gmail.com>> wrote:

Re: [ceph-users] CEPH RBD with ESXi

Hi Nikil. We just posted slides from Ceph Day (Los Angeles) about the use of iSCSI with Ceph at Electronic Arts. The slides can be found here if you want to review them. (Note: it doesn’t answer your specific question, but m

Re: [ceph-users] EC cluster design considerations

On Jul 4, 2015, at 2:44 PM, Adrien Gillard mailto:gillard.adr...@gmail.com>> wrote: Lastly, regarding Cluster Throughput: EC seems to require a bit more CPU and memory than straight replication, which begs the question of how much RAM and CPU are you putting into the chassis? With proper amou

Re: [ceph-users] EC cluster design considerations

HI Adrien. I can offer some feedback, and have a couple of questions myself: 1) if you’re going to deploy 9x4TB OSDs per host, with 7 hosts, and 4+2 EC, do you really want to put extras OSDs in ‘inner” drive bays if the target capacity is 100TB? My rough calculations indicate 150TB usable ca

Re: [ceph-users] Multiple journals and an OSD on one SSD doable?

Cameron, Somnath already covered most of these points, but I’ll add my $.02… The key question to me is this: will these 1TB SSDs perform well as a Journal target for Ceph? They’ll need to be fast at synchronous writes to fill that role, and if they aren’t I would use them for other OSD-relate

Re: [ceph-users] bursty IO, ceph cache pool can not follow evictions

Kenneth, My guess is that you’re hitting the cache_target_full_ratio on an individual OSD, which is easy to do since most of us tend to think of the cache_target_full_ratio as an aggregate of the OSDs (which it is not according to Greg Farnum). This posting may shed more light on the issue,

Re: [ceph-users] Kicking 'Remapped' PGs

It brings some comfort to know you found it weird too. In the end, we noted that the tunables were in ‘legacy’ mode - a hold over from prior experimentation, and a possible source of how we ended up with the remapped PGs in the first place. Setting that back to ‘firefly’ cleared up the remainin

Re: [ceph-users] Kicking 'Remapped' PGs

Gregory Farnum mailto:g...@gregs42.com>> wrote: Oh. That's strange; they are all mapped to two OSDs but are placed on two different ones. I'm...not sure why that would happen. Are these PGs active? What's the full output of "ceph -s"? Those 4 PG’s went inactive at some point, and we had the luxu

Re: [ceph-users] Kicking 'Remapped' PGs

Thanks, Greg. Following your lead, we discovered the proper 'set_choose_tries xxx’ value had not been applied to *this* pool’s rule, and we updated the cluster accordingly. We then moved a random OSD out and back in to ‘kick’ things, but no joy: we still have the 4 ‘remapped’ PGs. BTW: the 4 P

[ceph-users] Kicking 'Remapped' PGs

force_create_pg . Is this a viable command to use in a ‘remapped’ situation? Paul Evans ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Interesting problem: 2 pgs stuck in EC pool with missing OSDs

ster, while also changing the tunable ‘choose_total_tries’ to 150 from 50 (without that bump it seemed that we could still get a bad mapping). It only took a few minutes for the remaining 2 PG’s to successfully re-distribute their data, and we have finally reached HEALTH_OK. Thanks! -- Paul Evans

Re: [ceph-users] Recovering incomplete PGs with ceph_objectstore_tool

ht that that wouldn't > matter because OSD.15 did start and so we should have had everything but a > ceph pg query of the PGs showed something like: > "blocked": "peering is blocked due to down osds", > "down_osds_we_would_probe": [0], > "peering_blocked_by": [{ > "osd": 0, > "current_lost_at": 0, > "comment": "starting or marking this osd lost may let us proceed" > }] > > So I then removed OSD.0 from the cluster and everything came back to life. > Thanks to Jean-Charles Lopez, Craig Lewis, and Paul Evans! ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Interesting problem: 2 pgs stuck in EC pool with missing OSDs

and the EC pools seem to like more RAM that replicated pools do. Considering the RAM issue, is adding 2-4 OSDs at a time the recommendation? (other than adding more RAM). -- Paul Evans This looks like it's just the standard risk of using a pseudo-random algorithm: you need to "rando