Bo The problem with small files is that the TSM database entry may well be larger than the file you are storing. If your files are less than about 3000 bytes that will be the case.
What is happening is that the file system is being used as a database. A complex file path becomes the key and the file content is the data. I realize this has probably been dumped on you without consultation, but a database is probably a better fit. It could be something as simple as a key/value store (maybe one per day) or as complex as a document DB like Couchbase. A previous customer of mine did something similar. It was logs of ecommerce transactions that averaged about 1500 bytes each and had to be kept for 7 years. A million transactions a day and growing. They killed a TSM 5.5 database in 2 years, and when I left were well on the way to killing a TSM 6.3 database as well. Any requests to alter the application were met with active hostility. Good Luck Steve Steven Harris TSM Admin/Consultant Canberra Australia -----Original Message----- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Rick Adamson Sent: Tuesday, 21 March 2017 12:08 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] How to backup Billions of files ?? Bo I suggest you provide a few more details about the data and you backup environment. For example; what is this data, how frequently will it be accessed on average, what is its total space requirements, what is the source stored on? Type of backup storage; tape, disk, cloud? (specifics) Bandwidth/network speed between data and target backup server? -Rick Adamson Jacksonville,Fl. -----Original Message----- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Bo Nielsen Sent: Monday, March 20, 2017 7:20 AM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] How to backup Billions of files ?? Hi TSM's I have earlier asked for help with archiving of 80 Billion very small files, But now they want the files backed up. They expect an average change rate of 3 percent/Month. Anyone with experience of such an exercise, and will share it with me?? Regards Bo Bo Nielsen IT Service Technical University of Denmark IT Service Frederiksborgvej 399 Building 109 DK - 4000 Roskilde Denmark Mobil +45 2337 0271 boa...@dtu.dk<mailto:boa...@dtu.dk> This message and any attachment is confidential and may be privileged or otherwise protected from disclosure. You should immediately delete the message if you are not the intended recipient. If you have received this email by mistake please delete it from your system; you should not copy the message or disclose its content to anyone. This electronic communication may contain general financial product advice but should not be relied upon or construed as a recommendation of any financial product. The information has been prepared without taking into account your objectives, financial situation or needs. You should consider the Product Disclosure Statement relating to the financial product and consult your financial adviser before making a decision about whether to acquire, hold or dispose of a financial product. For further details on the financial product please go to http://www.bt.com.au Past performance is not a reliable indicator of future performance.