[ceph-users] why does 3 copies take so much more time than 2?

2023-01-04 Thread Charles Hedrick
I'm testing cephfs. I have 3 nodes, with 2 hard disks and one ssd on each. cephfs is set to put metadata on ssd and data on hdd. With the two pools set size = 3, untar'ing a 19 G file with 90K files in it takes 4.5 minutes. With size = 2, it takes 40 sec. (The tar file is stored in a file

[ceph-users] backups

2022-12-22 Thread Charles Hedrick
We have a ZFS file system with a billion (smallish) files. We backup using zfs send / receive to a separate system, and write tapes with zfs send. It stores files on HDD, but metadata on SSD. It would be totally impractical to backup to tape using something like tar from HDD with that many

[ceph-users] Re: what happens if a server crashes with cephfs?

2022-12-08 Thread Charles Hedrick
thanks. That's the behavior I was hoping for. From: Gregory Farnum Sent: Thursday, December 8, 2022 12:57 PM To: Charles Hedrick Cc: Manuel Holtgrewe ; Dhairya Parmar ; ceph-users@ceph.io Subject: Re: [ceph-users] Re: what happens if a server crashes

[ceph-users] Re: what happens if a server crashes with cephfs?

2022-12-08 Thread Charles Hedrick
for an operation that won't actually be possible to complete. I'm assuming that it won't happen during a cephadm upgrade. From: Manuel Holtgrewe Sent: Thursday, December 8, 2022 12:38 PM To: Charles Hedrick Cc: Gregory Farnum ; Dhairya Parmar ; ceph-users@ceph.io

[ceph-users] Re: what happens if a server crashes with cephfs?

2022-12-08 Thread Charles Hedrick
upgrade? Is that done in a way that won't generate errors in user code? From: Gregory Farnum Sent: Thursday, December 8, 2022 11:44 AM To: Manuel Holtgrewe Cc: Charles Hedrick ; Dhairya Parmar ; ceph-users@ceph.io Subject: Re: [ceph-users] Re: what happens

[ceph-users] Re: what happens if a server crashes with cephfs?

2022-12-08 Thread Charles Hedrick
reboot, and then continue. However there's an obvious performance penalty for this. From: Gregory Farnum Sent: Thursday, December 8, 2022 2:08 AM To: Dhairya Parmar Cc: Charles Hedrick ; ceph-users@ceph.io Subject: Re: [ceph-users] Re: what happens if a server

[ceph-users] what happens if a server crashes with cephfs?

2022-12-07 Thread Charles Hedrick
I believe asynchronous operations are used for some operations in cephfs. That means the server acknowledges before data has been written to stable storage. Does that mean there are failure scenarios when a write or close will return an error? fail silently?