Hi,
That's interesting.
What is the motivation/main benefits of the Panasas system? (what else did you
consider)
Yes inode trawling becoming a big issue.
----
Am wondering if it might be possible to [sensibly] do full (e.g. monthly) to
tape and incremental to disk (e.g. weekly).
David
________________________________
From: C. Chan <c-c...@uchicago.edu>
Sent: 24 May 2021 17:06
To: David Simpson <simpso...@cardiff.ac.uk>; amanda-users@amanda.org
<amanda-users@amanda.org>
Subject: Re: scale of backup and frequency
External email to Cardiff University - Take care when replying/opening
attachments or links.
Nid ebost mewnol o Brifysgol Caerdydd yw hwn - Cymerwch ofal wrth ateb/agor
atodiadau neu ddolenni.
FYI:
i. Three different backup sets, each around 100TB and growing.
ii. Fri evening - Mon morning weekend backups, rotated quarterly/every 12 weeks.
Filesystem snapshots done hourly, rotated each day, and daily snapshots,
rotated
each week, in place of daily incremental backups.
iii. The two biggest challenges are the explosion of the total volume of data
and the increase in IOPS limited small file I/O, both mostly due to deep
learning.
We are currently using NAS servers with ZFS special devices to store
small files
on NVMe SSD drives, but are planning to migrate/consolidate to a Panasas
system.
Also Sprach David Simpson:
> Hi all,
>
> Am interested to hear about your Amanda setup. Particularly if you are
> dealing with an HPC home file system/server, where both size and churn can be
> issues.
>
> And/or scalable storage.
>
> i) How big is your regular backup?
> ii) How frequent?
> iii) What challenges do you have/face?
>
> I'm currently in the process of thinking about how a new backup regime will
> look, armed with a new 40 slot tape library and some limited disk storage too.
>
> thanks
> David
>
--
C. Chan <c-chan at uchicago.edu>
GPG Public Key registered at pgp.mit.edu
