We've had this same issue with characters that are fine in Scale but Protect can't handle. Normally its because some script has embedded a newline in the middle of a file name, and normally we end up renaming that file by inode number
find . -inum 9975226749 -exec mv {} badfilename \; mostly because we can't even type the filename at the command prompt. However its not always just new line characters currently we've got a few files with unprintable characters in it. but its normally less than 50 files every few months, so is easy to handle manually. I normally end up looking at /data/mmbackup.unsupported which is the standard output from mmapplypolicy and extracting the file names from it and emailing the users concerned to assist them in working out what went wrong. I guess you could automate the parsing of this file at the end of the backup process and do something interesting with it. Peter Childs ________________________________________ From: gpfsug-discuss-boun...@spectrumscale.org <gpfsug-discuss-boun...@spectrumscale.org> on behalf of Simon Thompson <s.j.thomp...@bham.ac.uk> Sent: Monday, October 11, 2021 9:35 AM To: gpfsug main discussion list Subject: [EXTERNAL] Re: [gpfsug-discuss] Handling bad file names in policies? CAUTION: This email originated from outside of QMUL. Do not click links or open attachments unless you recognise the sender and know the content is safe. We have both: WILDCARDSARELITERAL yes QUOTESARELITERAL yes Set. And use --noquote for mmbackup, the backup runs, but creates a file: /filesystem/mmbackup.unsupported.CLIENTNAME Which contains a list of files that are not backed up due to \n in the filename. So it doesn't break backup, but they don't get backed up either. I believe this is because the TSM client can't back the file up rather than mmbackup no longer allowing them. I had an RFE at some point to get dsmc changed ... but it got closed WONTFIX. Simon On 09/10/2021, 10:09, "gpfsug-discuss-boun...@spectrumscale.org on behalf of Jonathan Buzzard" <gpfsug-discuss-boun...@spectrumscale.org on behalf of jonathan.buzz...@strath.ac.uk> wrote: On 08/10/2021 19:14, Wahl, Edward wrote: > This goes back as far as I can recall to <=GPFS 3.5 days. And no, I > cannot recall what version of TSM-EE that was. But newline has been > the only stopping point, for what seems like forever. Having filed > many an mmbackup bug, I don't recall ever crashing on filenames. > (tons of OTHER reasons, but not character set) We even generate an > error report from this and email users to fix it. We accept basically > almost everything else, and I have to say, we see some really crazy > things sometimes. I think my current favorite is the full windows > paths as a filename. (eg: > "Y:\Temp\temp\290\work\0\Material_ERTi-5.in" ) > I will have to do a test but I am sure newlines have worked just fine in the past. At the very least they have not stopped an entire backup from working when using dsmc incr. Now mmbackup that's a different kettle of fish. If you have not seen mmbackup fail entirely because of a random "special" character you simply have not been using it long enough :-) For the longest of times I would simply not go anywhere near it because it was not fit for purpose. > > Current IBM documentation doesn't go backwards past 4.2 but it says: > > "For IBM Spectrum Scale™ file systems with special characters > frequently used in the names of files or directories, backup failures > might occur. Known special characters that require special handling > include: *, ?, ", ’, carriage return, and the new line character. > > In such cases, enable the Tivoli Storage Manager client options > WILDCARDSARELITERAL and QUOTESARELITERAL on all nodes that are used > in backup activities and make sure that the mmbackup option --noquote > is used when invoking mmbackup." > > So maybe we could handle newlines somehow. But my lazy searches > didn't show what TSM doesn't accept. > We strongly advise our users (our GPFS file system is for an HPC system) in training not to use "special" characters. That is followed with a warning that if they do then we don't make any promises to backup their files :-) From time to time I run a dsmc incr in a screen and capture the output to a log file and then look at the list of failed files and prompt users to "fix" them. Though sometimes I just "fix" them myself if the correction is going to be obvious and then email them to tell them what has happened. JAB. -- Jonathan A. Buzzard Tel: +44141-5483420 HPC System Administrator, ARCHIE-WeSt. University of Strathclyde, John Anderson Building, Glasgow. G4 0NG _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss