We've had this same issue with characters that are fine in Scale but Protect
can't handle. Normally its because some script has embedded a newline in the
middle of a file name, and normally we end up renaming that file by inode number
find . -inum 9975226749 -exec mv {} badfilename \;
mostly because we can't even type the filename at the command prompt.
However its not always just new line characters currently we've got a few files
with unprintable characters in it. but its normally less than 50 files every
few months, so is easy to handle manually.
I normally end up looking at /data/mmbackup.unsupported which is the standard
output from mmapplypolicy and extracting the file names from it and emailing
the users concerned to assist them in working out what went wrong.
I guess you could automate the parsing of this file at the end of the backup
process and do something interesting with it.
Peter Childs
________________________________________
From: gpfsug-discuss-boun...@spectrumscale.org
<gpfsug-discuss-boun...@spectrumscale.org> on behalf of Simon Thompson
<s.j.thomp...@bham.ac.uk>
Sent: Monday, October 11, 2021 9:35 AM
To: gpfsug main discussion list
Subject: [EXTERNAL] Re: [gpfsug-discuss] Handling bad file names in policies?
CAUTION: This email originated from outside of QMUL. Do not click links or open
attachments unless you recognise the sender and know the content is safe.
We have both:
WILDCARDSARELITERAL yes
QUOTESARELITERAL yes
Set. And use --noquote for mmbackup, the backup runs, but creates a file:
/filesystem/mmbackup.unsupported.CLIENTNAME
Which contains a list of files that are not backed up due to \n in the filename.
So it doesn't break backup, but they don't get backed up either. I believe this
is because the TSM client can't back the file up rather than mmbackup no longer
allowing them. I had an RFE at some point to get dsmc changed ... but it got
closed WONTFIX.
Simon
On 09/10/2021, 10:09, "gpfsug-discuss-boun...@spectrumscale.org on behalf of
Jonathan Buzzard" <gpfsug-discuss-boun...@spectrumscale.org on behalf of
jonathan.buzz...@strath.ac.uk> wrote:
On 08/10/2021 19:14, Wahl, Edward wrote:
> This goes back as far as I can recall to <=GPFS 3.5 days. And no, I
> cannot recall what version of TSM-EE that was. But newline has been
> the only stopping point, for what seems like forever. Having filed
> many an mmbackup bug, I don't recall ever crashing on filenames.
> (tons of OTHER reasons, but not character set) We even generate an
> error report from this and email users to fix it. We accept basically
> almost everything else, and I have to say, we see some really crazy
> things sometimes. I think my current favorite is the full windows
> paths as a filename. (eg:
> "Y:\Temp\temp\290\work\0\Material_ERTi-5.in" )
>
I will have to do a test but I am sure newlines have worked just fine in
the past. At the very least they have not stopped an entire backup from
working when using dsmc incr.
Now mmbackup that's a different kettle of fish. If you have not seen
mmbackup fail entirely because of a random "special" character you
simply have not been using it long enough :-)
For the longest of times I would simply not go anywhere near it because
it was not fit for purpose.
>
> Current IBM documentation doesn't go backwards past 4.2 but it says:
>
> "For IBM Spectrum Scale™ file systems with special characters
> frequently used in the names of files or directories, backup failures
> might occur. Known special characters that require special handling
> include: *, ?, ", ’, carriage return, and the new line character.
>
> In such cases, enable the Tivoli Storage Manager client options
> WILDCARDSARELITERAL and QUOTESARELITERAL on all nodes that are used
> in backup activities and make sure that the mmbackup option --noquote
> is used when invoking mmbackup."
>
> So maybe we could handle newlines somehow. But my lazy searches
> didn't show what TSM doesn't accept.
>
We strongly advise our users (our GPFS file system is for an HPC system)
in training not to use "special" characters. That is followed with a
warning that if they do then we don't make any promises to backup their
files :-)
From time to time I run a dsmc incr in a screen and capture the output
to a log file and then look at the list of failed files and prompt users
to "fix" them. Though sometimes I just "fix" them myself if the
correction is going to be obvious and then email them to tell them what
has happened.
JAB.
--
Jonathan A. Buzzard Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
