Re: Linux Client failing with ANS1999E error
Andrew, Thanks for your help. '/public/proval-old' was the culprit. We are now backing up without errors. -Original Message- From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On Behalf Of Andrew Raibeck Sent: Tuesday, April 20, 2010 6:17 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: Linux Client failing with ANS1999E error I suspect the error occurs when the backup-archive client attempts to stat the file system. Are you able to successfully "stat" this file system? stat /public/proval-old Andy Raibeck IBM Software Group Tivoli Storage Manager Client Product Development Level 3 Team Lead Internal Notes e-mail: Andrew Raibeck/Hartford/i...@ibmus Internet e-mail: stor...@us.ibm.com IBM Tivoli Storage Manager support web page: http://www.ibm.com/support/entry/portal/Overview/Software/Tivoli/Tivoli_Storage_Manager The only dumb question is the one that goes unasked. The command line is your friend. "Good enough" is the enemy of excellence. "ADSM: Dist Stor Manager" wrote on 2010-04-19 17:40:57: > [image removed] > > Re: Linux Client failing with ANS1999E error > > James Choate > > to: > > ADSM-L > > 2010-04-19 21:09 > > Sent by: > > "ADSM: Dist Stor Manager" > > Please respond to "ADSM: Dist Stor Manager" > > I just ran an fsck on /public/proval-old, No errors were found on that mount. > > -Original Message- > From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On > Behalf Of Ben Bullock > Sent: Monday, April 19, 2010 10:25 AM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: Linux Client failing with ANS1999E error > > I have seen something like this before. I would guess that according > to the log, that something is corrupt in '/public/proval-old' . > Perhaps a bad inode, invalid file name or just plain corrupt file. I > would do some 'ls -l' in and around that directory looking for a bad file. > > TSM backups are great for sniffing out filesystem problems. ;-) > > Ben > > > -Original Message- > From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On > Behalf Of James Choate > Sent: Monday, April 19, 2010 9:46 AM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: [ADSM-L] Linux Client failing with ANS1999E error > > 04/19/2010 08:38:26 > > 04/19/2010 08:38:26 Schedule Name: TEST_SCHEDULE > 04/19/2010 08:38:26 Action:Incremental > 04/19/2010 08:38:26 Objects: > 04/19/2010 08:38:26 Options: > 04/19/2010 08:38:26 Server Window Start: 08:40:00 on 04/19/2010 > 04/19/2010 08:38:26 > > 04/19/2010 08:38:26 Command will be executed in 18 minutes. > 04/19/2010 08:56:26 > Executing scheduled command now. > 04/19/2010 08:56:26 Node Name: CLLINUX1 04/19/2010 08:56:26 Session > established with server TSMSRV1: Windows > 04/19/2010 08:56:26 Server Version 5, Release 5, Level 0.0 > 04/19/2010 08:56:26 Server date/time: 04/19/2010 08:56:26 Last > access: 04/19/2010 08:38:26 > > 04/19/2010 08:56:26 --- SCHEDULEREC OBJECT BEGIN TEST_SCHEDULE 04/ > 19/2010 08:40:00 04/19/2010 08:56:27 Incremental backup of volume '/' > 04/19/2010 08:56:27 Incremental backup of volume '/boot' > 04/19/2010 08:56:27 Incremental backup of volume '/public' > 04/19/2010 08:56:27 Incremental backup of volume '/pictometry' > 04/19/2010 08:56:27 Incremental backup of volume '/public/proval-old' > 04/19/2010 08:56:28 ANS1115W File '/public/proval-old' excluded by > Include/Exclude list 04/19/2010 08:56:28 Successful incremental > backup of '/public/proval-old' > > 04/19/2010 08:56:28 ANS1999E Incremental processing of '/' stopped. > > 04/19/2010 08:56:28 --- SCHEDULEREC STATUS BEGIN > 04/19/2010 08:56:28 Total number of objects inspected:1 > 04/19/2010 08:56:28 Total number of objects backed up:0 > 04/19/2010 08:56:28 Total number of objects updated: 0 > 04/19/2010 08:56:28 Total number of objects rebound: 0 > 04/19/2010 08:56:28 Total number of objects deleted: 0 > 04/19/2010 08:56:28 Total number of objects expired: 0 > 04/19/2010 08:56:28 Total number of objects failed: 0 > 04/19/2010 08:56:28 Total number of bytes transferred: 0 B > 04/19/2010 08:56:28 Data transfer time:0.00 sec > 04/19/2010 08:56:28 Network data transfer rate:0.00 KB/sec > 04/19/2010 08:56:28 Aggregate data transfer rate: 0.00 KB/sec > 04/19/2010 08:56:28 Objects compressed by:0% > 04/19/2010 08:56:28 Elapsed processing time: 0
Re: Linux Client failing with ANS1999E error
>> On Tue, 20 Apr 2010 08:39:11 -0400, Lindsay Morris >> said: > The TSM admin is like the canary in the coal mine. > (Wait, that's a terrible analogy .. they end up dead...) Terrible because it's inaccurate, or terrible because we don't like the implications? Your viewers want to know. :) - Allen S. Rout
Re: Linux Client failing with ANS1999E error
Ben said TSM backups are great for sniffing out filesystem problems. ;-) I would add "and network problems, and application problems , and database performance problems ..." The TSM admin is like the canary in the coal mine. (Wait, that's a terrible analogy .. they end up dead...) Lindsay Morris CEO, TSMworks Tel. 1-859-539-9900 lind...@tsmworks.com
Re: Linux Client failing with ANS1999E error
I suspect the error occurs when the backup-archive client attempts to stat the file system. Are you able to successfully "stat" this file system? stat /public/proval-old Andy Raibeck IBM Software Group Tivoli Storage Manager Client Product Development Level 3 Team Lead Internal Notes e-mail: Andrew Raibeck/Hartford/i...@ibmus Internet e-mail: stor...@us.ibm.com IBM Tivoli Storage Manager support web page: http://www.ibm.com/support/entry/portal/Overview/Software/Tivoli/Tivoli_Storage_Manager The only dumb question is the one that goes unasked. The command line is your friend. "Good enough" is the enemy of excellence. "ADSM: Dist Stor Manager" wrote on 2010-04-19 17:40:57: > [image removed] > > Re: Linux Client failing with ANS1999E error > > James Choate > > to: > > ADSM-L > > 2010-04-19 21:09 > > Sent by: > > "ADSM: Dist Stor Manager" > > Please respond to "ADSM: Dist Stor Manager" > > I just ran an fsck on /public/proval-old, No errors were found on that mount. > > -Original Message- > From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On > Behalf Of Ben Bullock > Sent: Monday, April 19, 2010 10:25 AM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: Linux Client failing with ANS1999E error > > I have seen something like this before. I would guess that according > to the log, that something is corrupt in '/public/proval-old' . > Perhaps a bad inode, invalid file name or just plain corrupt file. I > would do some 'ls -l' in and around that directory looking for a bad file. > > TSM backups are great for sniffing out filesystem problems. ;-) > > Ben > > > -Original Message----- > From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On > Behalf Of James Choate > Sent: Monday, April 19, 2010 9:46 AM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: [ADSM-L] Linux Client failing with ANS1999E error > > 04/19/2010 08:38:26 > > 04/19/2010 08:38:26 Schedule Name: TEST_SCHEDULE > 04/19/2010 08:38:26 Action:Incremental > 04/19/2010 08:38:26 Objects: > 04/19/2010 08:38:26 Options: > 04/19/2010 08:38:26 Server Window Start: 08:40:00 on 04/19/2010 > 04/19/2010 08:38:26 > > 04/19/2010 08:38:26 Command will be executed in 18 minutes. > 04/19/2010 08:56:26 > Executing scheduled command now. > 04/19/2010 08:56:26 Node Name: CLLINUX1 04/19/2010 08:56:26 Session > established with server TSMSRV1: Windows > 04/19/2010 08:56:26 Server Version 5, Release 5, Level 0.0 > 04/19/2010 08:56:26 Server date/time: 04/19/2010 08:56:26 Last > access: 04/19/2010 08:38:26 > > 04/19/2010 08:56:26 --- SCHEDULEREC OBJECT BEGIN TEST_SCHEDULE 04/ > 19/2010 08:40:00 04/19/2010 08:56:27 Incremental backup of volume '/' > 04/19/2010 08:56:27 Incremental backup of volume '/boot' > 04/19/2010 08:56:27 Incremental backup of volume '/public' > 04/19/2010 08:56:27 Incremental backup of volume '/pictometry' > 04/19/2010 08:56:27 Incremental backup of volume '/public/proval-old' > 04/19/2010 08:56:28 ANS1115W File '/public/proval-old' excluded by > Include/Exclude list 04/19/2010 08:56:28 Successful incremental > backup of '/public/proval-old' > > 04/19/2010 08:56:28 ANS1999E Incremental processing of '/' stopped. > > 04/19/2010 08:56:28 --- SCHEDULEREC STATUS BEGIN > 04/19/2010 08:56:28 Total number of objects inspected:1 > 04/19/2010 08:56:28 Total number of objects backed up:0 > 04/19/2010 08:56:28 Total number of objects updated: 0 > 04/19/2010 08:56:28 Total number of objects rebound: 0 > 04/19/2010 08:56:28 Total number of objects deleted: 0 > 04/19/2010 08:56:28 Total number of objects expired: 0 > 04/19/2010 08:56:28 Total number of objects failed: 0 > 04/19/2010 08:56:28 Total number of bytes transferred: 0 B > 04/19/2010 08:56:28 Data transfer time:0.00 sec > 04/19/2010 08:56:28 Network data transfer rate:0.00 KB/sec > 04/19/2010 08:56:28 Aggregate data transfer rate: 0.00 KB/sec > 04/19/2010 08:56:28 Objects compressed by:0% > 04/19/2010 08:56:28 Elapsed processing time: 00:00:01 > 04/19/2010 08:56:28 --- SCHEDULEREC STATUS END 04/19/2010 08:56:28 > ANS1028S An internal program error occurred. > 04/19/2010 08:56:28 --- SCHEDULEREC OBJECT END TEST_SCHEDULE 04/19/ > 2010 08:40:00 04/19/2010 08:56:28 ANS1512E Scheduled event > 'TEST_SCHEDULE' failed. > Return code = 12. > 04/19/2010 08:56:28
Re: Linux Client failing with ANS1999E error
On Apr 19, 2010, at 5:41 PM, James Choate wrote: > I am looking for the longest fully qualified file name on the system. > > An error=6 might be indicating ENAMETOOLONG (The only errno that equals 6 in > Linux). Errno 6 is ENXIO. FWIW, I've accumulated some notes on it over time... ENXIO 6 No such device or address. Typically, this means that a file representing a device has been installed incorrectly, and the system can't find the right kind of device driver for it. Is expected if opening a FIFO for writing with the O_NDELAY flag, thus indicating that no process yet has the FIFO open for reading. I/O on a special file refers to a subdevice that does not exist, or beyond the limits of the device. It may also occur when, for example, an illegal tape drive unit number is selected or a disk pack is not loaded on a drive. Also seen when trying to open /dev/lmcp0 and the /etc/lmcpd daemon is not running. When the operation involves access to memory areas (e.g., /dev/kmem) it may be that the address/offset being used is out of range. (Interestingly, an lseek() or lseek64() to a bogus address will not complain, but the following read() which depends upon that seek will fail.) Richard Simshttp://people.bu.edu/rbs/
Re: Linux Client failing with ANS1999E error
I just ran an fsck on /public/proval-old, No errors were found on that mount. -Original Message- From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On Behalf Of Ben Bullock Sent: Monday, April 19, 2010 10:25 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: Linux Client failing with ANS1999E error I have seen something like this before. I would guess that according to the log, that something is corrupt in '/public/proval-old' . Perhaps a bad inode, invalid file name or just plain corrupt file. I would do some 'ls -l' in and around that directory looking for a bad file. TSM backups are great for sniffing out filesystem problems. ;-) Ben -Original Message- From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On Behalf Of James Choate Sent: Monday, April 19, 2010 9:46 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Linux Client failing with ANS1999E error 04/19/2010 08:38:26 04/19/2010 08:38:26 Schedule Name: TEST_SCHEDULE 04/19/2010 08:38:26 Action:Incremental 04/19/2010 08:38:26 Objects: 04/19/2010 08:38:26 Options: 04/19/2010 08:38:26 Server Window Start: 08:40:00 on 04/19/2010 04/19/2010 08:38:26 04/19/2010 08:38:26 Command will be executed in 18 minutes. 04/19/2010 08:56:26 Executing scheduled command now. 04/19/2010 08:56:26 Node Name: CLLINUX1 04/19/2010 08:56:26 Session established with server TSMSRV1: Windows 04/19/2010 08:56:26 Server Version 5, Release 5, Level 0.0 04/19/2010 08:56:26 Server date/time: 04/19/2010 08:56:26 Last access: 04/19/2010 08:38:26 04/19/2010 08:56:26 --- SCHEDULEREC OBJECT BEGIN TEST_SCHEDULE 04/19/2010 08:40:00 04/19/2010 08:56:27 Incremental backup of volume '/' 04/19/2010 08:56:27 Incremental backup of volume '/boot' 04/19/2010 08:56:27 Incremental backup of volume '/public' 04/19/2010 08:56:27 Incremental backup of volume '/pictometry' 04/19/2010 08:56:27 Incremental backup of volume '/public/proval-old' 04/19/2010 08:56:28 ANS1115W File '/public/proval-old' excluded by Include/Exclude list 04/19/2010 08:56:28 Successful incremental backup of '/public/proval-old' 04/19/2010 08:56:28 ANS1999E Incremental processing of '/' stopped. 04/19/2010 08:56:28 --- SCHEDULEREC STATUS BEGIN 04/19/2010 08:56:28 Total number of objects inspected:1 04/19/2010 08:56:28 Total number of objects backed up:0 04/19/2010 08:56:28 Total number of objects updated: 0 04/19/2010 08:56:28 Total number of objects rebound: 0 04/19/2010 08:56:28 Total number of objects deleted: 0 04/19/2010 08:56:28 Total number of objects expired: 0 04/19/2010 08:56:28 Total number of objects failed: 0 04/19/2010 08:56:28 Total number of bytes transferred: 0 B 04/19/2010 08:56:28 Data transfer time:0.00 sec 04/19/2010 08:56:28 Network data transfer rate:0.00 KB/sec 04/19/2010 08:56:28 Aggregate data transfer rate: 0.00 KB/sec 04/19/2010 08:56:28 Objects compressed by:0% 04/19/2010 08:56:28 Elapsed processing time: 00:00:01 04/19/2010 08:56:28 --- SCHEDULEREC STATUS END 04/19/2010 08:56:28 ANS1028S An internal program error occurred. 04/19/2010 08:56:28 --- SCHEDULEREC OBJECT END TEST_SCHEDULE 04/19/2010 08:40:00 04/19/2010 08:56:28 ANS1512E Scheduled event 'TEST_SCHEDULE' failed. Return code = 12. 04/19/2010 08:56:28 Sending results for scheduled event 'TEST_SCHEDULE'. 04/19/2010 08:56:28 Results sent to server for scheduled event 'TEST_SCHEDULE'. 04/19/2010 08:56:28 ANS1483I Schedule log pruning started. 04/19/2010 08:56:28 ANS1484I Schedule log pruning finished successfully. 04/19/2010 08:56:28 TSM Backup-Archive Client Version 5, Release 5, Level 2.0 04/19/2010 08:56:28 Querying server for next scheduled event. 04/19/2010 08:56:28 Node Name: CLLINUX1 04/19/2010 08:56:28 Session established with server TSMSRV1: Windows 04/19/2010 08:56:28 Server Version 5, Release 5, Level 0.0 04/19/2010 08:56:28 Server date/time: 04/19/2010 08:56:28 Last access: 04/19/2010 08:56:28 04/19/2010 08:56:28 --- SCHEDULEREC QUERY BEGIN 04/19/2010 08:56:28 --- SCHEDULEREC QUERY END 04/19/2010 08:56:28 Next operation scheduled: 04/19/2010 08:56:28 04/19/2010 08:56:28 Schedule Name: TEST_SCHEDULE 04/19/2010 08:56:28 Action:Incremental 04/19/2010 08:56:28 Objects: 04/19/2010 08:56:28 Options: 04/19/2010 08:56:28 Server Window Start: 08:40:00 on 04/20/2010 04/19/2010 08:56:28 04/19/2010 08:56:28 Schedule will be refreshed in 6 hours. This is the info from dmserror.log for this run. 04/19/2010 08:35:34 ANS2820E An interrupt h
Re: Linux Client failing with ANS1999E error
I am looking for the longest fully qualified file name on the system. An error=6 might be indicating ENAMETOOLONG (The only errno that equals 6 in Linux). Repeated runs (manual) today have resulting in the exact same result. -Original Message- From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On Behalf Of Richard Sims Sent: Monday, April 19, 2010 10:31 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: Linux Client failing with ANS1999E error On Apr 19, 2010, at 11:45 AM, James Choate wrote: > 04/19/2010 08:56:26 --- SCHEDULEREC OBJECT BEGIN TEST_SCHEDULE > 04/19/2010 08:40:00 04/19/2010 08:56:27 Incremental backup of volume > '/' > 04/19/2010 08:56:27 Incremental backup of volume '/boot' > 04/19/2010 08:56:27 Incremental backup of volume '/public' > 04/19/2010 08:56:27 Incremental backup of volume '/pictometry' > 04/19/2010 08:56:27 Incremental backup of volume '/public/proval-old' > 04/19/2010 08:56:28 ANS1115W File '/public/proval-old' excluded by > Include/Exclude list 04/19/2010 08:56:28 Successful incremental > backup of '/public/proval-old' Jim - In my April 16th response I suggested that the /public/proval-old file system object be examined to determine exactly what it is, and if there is anything wrong with it. That still needs to be done. It is very odd that it should be included in "Incremental backup of volume" messages, as those reflect Domain statements in your client options; and you already have a /public in there, which on the surface of it would preclude having a /public/proval-old (whose name suggests that the word "approval" maybe should have been in there). And having / boot identified as a separate file system doesn't seem right for a Linux system, where /boot is normally a subdirectory in the root volume. I would recommend a full review of the client options (which may be on the TSM client and server), to remove inappropriate Virtualmountpoint specifications (along with proper obsolescence of the corresponding filespaces on the TSM server). Things just aren't right on that client system. Richard Sims
Re: Linux Client failing with ANS1999E error
It seems to me that /public/proval-old is a filesystem of some sort since there was no other virtualmountpoint messages. Maybe it's a loopback mount or something? Try putting this in there to exclude it: exclude.fs /public/proval-old If that doesn't work, provide the output of a "df" If it does work, it would just be a workaround for something weird going on with your filesystems. Regards, Shawn Shawn Drew Internet r...@bu.edu Sent by: ADSM-L@VM.MARIST.EDU 04/19/2010 12:31 PM Please respond to ADSM-L@VM.MARIST.EDU To ADSM-L cc Subject Re: [ADSM-L] Linux Client failing with ANS1999E error On Apr 19, 2010, at 11:45 AM, James Choate wrote: > 04/19/2010 08:56:26 --- SCHEDULEREC OBJECT BEGIN TEST_SCHEDULE > 04/19/2010 08:40:00 04/19/2010 08:56:27 Incremental backup of volume > '/' > 04/19/2010 08:56:27 Incremental backup of volume '/boot' > 04/19/2010 08:56:27 Incremental backup of volume '/public' > 04/19/2010 08:56:27 Incremental backup of volume '/pictometry' > 04/19/2010 08:56:27 Incremental backup of volume '/public/proval-old' > 04/19/2010 08:56:28 ANS1115W File '/public/proval-old' excluded by > Include/Exclude list 04/19/2010 08:56:28 Successful incremental > backup of '/public/proval-old' Jim - In my April 16th response I suggested that the /public/proval-old file system object be examined to determine exactly what it is, and if there is anything wrong with it. That still needs to be done. It is very odd that it should be included in "Incremental backup of volume" messages, as those reflect Domain statements in your client options; and you already have a /public in there, which on the surface of it would preclude having a /public/proval-old (whose name suggests that the word "approval" maybe should have been in there). And having / boot identified as a separate file system doesn't seem right for a Linux system, where /boot is normally a subdirectory in the root volume. I would recommend a full review of the client options (which may be on the TSM client and server), to remove inappropriate Virtualmountpoint specifications (along with proper obsolescence of the corresponding filespaces on the TSM server). Things just aren't right on that client system. Richard Sims This message and any attachments (the "message") is intended solely for the addressees and is confidential. If you receive this message in error, please delete it and immediately notify the sender. Any use not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. The internet can not guarantee the integrity of this message. BNP PARIBAS (and its subsidiaries) shall (will) not therefore be liable for the message if modified. Please note that certain functions and services for BNP Paribas may be performed by BNP Paribas RCC, Inc.
Re: Linux Client failing with ANS1999E error
On Apr 19, 2010, at 11:45 AM, James Choate wrote: 04/19/2010 08:56:26 --- SCHEDULEREC OBJECT BEGIN TEST_SCHEDULE 04/19/2010 08:40:00 04/19/2010 08:56:27 Incremental backup of volume '/' 04/19/2010 08:56:27 Incremental backup of volume '/boot' 04/19/2010 08:56:27 Incremental backup of volume '/public' 04/19/2010 08:56:27 Incremental backup of volume '/pictometry' 04/19/2010 08:56:27 Incremental backup of volume '/public/proval-old' 04/19/2010 08:56:28 ANS1115W File '/public/proval-old' excluded by Include/Exclude list 04/19/2010 08:56:28 Successful incremental backup of '/public/proval-old' Jim - In my April 16th response I suggested that the /public/proval-old file system object be examined to determine exactly what it is, and if there is anything wrong with it. That still needs to be done. It is very odd that it should be included in "Incremental backup of volume" messages, as those reflect Domain statements in your client options; and you already have a /public in there, which on the surface of it would preclude having a /public/proval-old (whose name suggests that the word "approval" maybe should have been in there). And having / boot identified as a separate file system doesn't seem right for a Linux system, where /boot is normally a subdirectory in the root volume. I would recommend a full review of the client options (which may be on the TSM client and server), to remove inappropriate Virtualmountpoint specifications (along with proper obsolescence of the corresponding filespaces on the TSM server). Things just aren't right on that client system. Richard Sims
Re: Linux Client failing with ANS1999E error
I have seen something like this before. I would guess that according to the log, that something is corrupt in '/public/proval-old' . Perhaps a bad inode, invalid file name or just plain corrupt file. I would do some 'ls -l' in and around that directory looking for a bad file. TSM backups are great for sniffing out filesystem problems. ;-) Ben -Original Message- From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On Behalf Of James Choate Sent: Monday, April 19, 2010 9:46 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Linux Client failing with ANS1999E error 04/19/2010 08:38:26 04/19/2010 08:38:26 Schedule Name: TEST_SCHEDULE 04/19/2010 08:38:26 Action:Incremental 04/19/2010 08:38:26 Objects: 04/19/2010 08:38:26 Options: 04/19/2010 08:38:26 Server Window Start: 08:40:00 on 04/19/2010 04/19/2010 08:38:26 04/19/2010 08:38:26 Command will be executed in 18 minutes. 04/19/2010 08:56:26 Executing scheduled command now. 04/19/2010 08:56:26 Node Name: CLLINUX1 04/19/2010 08:56:26 Session established with server TSMSRV1: Windows 04/19/2010 08:56:26 Server Version 5, Release 5, Level 0.0 04/19/2010 08:56:26 Server date/time: 04/19/2010 08:56:26 Last access: 04/19/2010 08:38:26 04/19/2010 08:56:26 --- SCHEDULEREC OBJECT BEGIN TEST_SCHEDULE 04/19/2010 08:40:00 04/19/2010 08:56:27 Incremental backup of volume '/' 04/19/2010 08:56:27 Incremental backup of volume '/boot' 04/19/2010 08:56:27 Incremental backup of volume '/public' 04/19/2010 08:56:27 Incremental backup of volume '/pictometry' 04/19/2010 08:56:27 Incremental backup of volume '/public/proval-old' 04/19/2010 08:56:28 ANS1115W File '/public/proval-old' excluded by Include/Exclude list 04/19/2010 08:56:28 Successful incremental backup of '/public/proval-old' 04/19/2010 08:56:28 ANS1999E Incremental processing of '/' stopped. 04/19/2010 08:56:28 --- SCHEDULEREC STATUS BEGIN 04/19/2010 08:56:28 Total number of objects inspected:1 04/19/2010 08:56:28 Total number of objects backed up:0 04/19/2010 08:56:28 Total number of objects updated: 0 04/19/2010 08:56:28 Total number of objects rebound: 0 04/19/2010 08:56:28 Total number of objects deleted: 0 04/19/2010 08:56:28 Total number of objects expired: 0 04/19/2010 08:56:28 Total number of objects failed: 0 04/19/2010 08:56:28 Total number of bytes transferred: 0 B 04/19/2010 08:56:28 Data transfer time:0.00 sec 04/19/2010 08:56:28 Network data transfer rate:0.00 KB/sec 04/19/2010 08:56:28 Aggregate data transfer rate: 0.00 KB/sec 04/19/2010 08:56:28 Objects compressed by:0% 04/19/2010 08:56:28 Elapsed processing time: 00:00:01 04/19/2010 08:56:28 --- SCHEDULEREC STATUS END 04/19/2010 08:56:28 ANS1028S An internal program error occurred. 04/19/2010 08:56:28 --- SCHEDULEREC OBJECT END TEST_SCHEDULE 04/19/2010 08:40:00 04/19/2010 08:56:28 ANS1512E Scheduled event 'TEST_SCHEDULE' failed. Return code = 12. 04/19/2010 08:56:28 Sending results for scheduled event 'TEST_SCHEDULE'. 04/19/2010 08:56:28 Results sent to server for scheduled event 'TEST_SCHEDULE'. 04/19/2010 08:56:28 ANS1483I Schedule log pruning started. 04/19/2010 08:56:28 ANS1484I Schedule log pruning finished successfully. 04/19/2010 08:56:28 TSM Backup-Archive Client Version 5, Release 5, Level 2.0 04/19/2010 08:56:28 Querying server for next scheduled event. 04/19/2010 08:56:28 Node Name: CLLINUX1 04/19/2010 08:56:28 Session established with server TSMSRV1: Windows 04/19/2010 08:56:28 Server Version 5, Release 5, Level 0.0 04/19/2010 08:56:28 Server date/time: 04/19/2010 08:56:28 Last access: 04/19/2010 08:56:28 04/19/2010 08:56:28 --- SCHEDULEREC QUERY BEGIN 04/19/2010 08:56:28 --- SCHEDULEREC QUERY END 04/19/2010 08:56:28 Next operation scheduled: 04/19/2010 08:56:28 04/19/2010 08:56:28 Schedule Name: TEST_SCHEDULE 04/19/2010 08:56:28 Action:Incremental 04/19/2010 08:56:28 Objects: 04/19/2010 08:56:28 Options: 04/19/2010 08:56:28 Server Window Start: 08:40:00 on 04/20/2010 04/19/2010 08:56:28 04/19/2010 08:56:28 Schedule will be refreshed in 6 hours. This is the info from dmserror.log for this run. 04/19/2010 08:35:34 ANS2820E An interrupt has occurred. The current operation will end and the client will shut down. 04/19/2010 08:36:12 ANS2820E An interrupt has occurred. The current operation will end and the client will shut down. 04/19/2010 08:36:49 ANS2820E An interrupt has occurred. The current operation will end and the client will shut
Re: Linux Client failing with ANS1999E error
04/19/2010 08:38:26 04/19/2010 08:38:26 Schedule Name: TEST_SCHEDULE 04/19/2010 08:38:26 Action:Incremental 04/19/2010 08:38:26 Objects: 04/19/2010 08:38:26 Options: 04/19/2010 08:38:26 Server Window Start: 08:40:00 on 04/19/2010 04/19/2010 08:38:26 04/19/2010 08:38:26 Command will be executed in 18 minutes. 04/19/2010 08:56:26 Executing scheduled command now. 04/19/2010 08:56:26 Node Name: CLLINUX1 04/19/2010 08:56:26 Session established with server TSMSRV1: Windows 04/19/2010 08:56:26 Server Version 5, Release 5, Level 0.0 04/19/2010 08:56:26 Server date/time: 04/19/2010 08:56:26 Last access: 04/19/2010 08:38:26 04/19/2010 08:56:26 --- SCHEDULEREC OBJECT BEGIN TEST_SCHEDULE 04/19/2010 08:40:00 04/19/2010 08:56:27 Incremental backup of volume '/' 04/19/2010 08:56:27 Incremental backup of volume '/boot' 04/19/2010 08:56:27 Incremental backup of volume '/public' 04/19/2010 08:56:27 Incremental backup of volume '/pictometry' 04/19/2010 08:56:27 Incremental backup of volume '/public/proval-old' 04/19/2010 08:56:28 ANS1115W File '/public/proval-old' excluded by Include/Exclude list 04/19/2010 08:56:28 Successful incremental backup of '/public/proval-old' 04/19/2010 08:56:28 ANS1999E Incremental processing of '/' stopped. 04/19/2010 08:56:28 --- SCHEDULEREC STATUS BEGIN 04/19/2010 08:56:28 Total number of objects inspected:1 04/19/2010 08:56:28 Total number of objects backed up:0 04/19/2010 08:56:28 Total number of objects updated: 0 04/19/2010 08:56:28 Total number of objects rebound: 0 04/19/2010 08:56:28 Total number of objects deleted: 0 04/19/2010 08:56:28 Total number of objects expired: 0 04/19/2010 08:56:28 Total number of objects failed: 0 04/19/2010 08:56:28 Total number of bytes transferred: 0 B 04/19/2010 08:56:28 Data transfer time:0.00 sec 04/19/2010 08:56:28 Network data transfer rate:0.00 KB/sec 04/19/2010 08:56:28 Aggregate data transfer rate: 0.00 KB/sec 04/19/2010 08:56:28 Objects compressed by:0% 04/19/2010 08:56:28 Elapsed processing time: 00:00:01 04/19/2010 08:56:28 --- SCHEDULEREC STATUS END 04/19/2010 08:56:28 ANS1028S An internal program error occurred. 04/19/2010 08:56:28 --- SCHEDULEREC OBJECT END TEST_SCHEDULE 04/19/2010 08:40:00 04/19/2010 08:56:28 ANS1512E Scheduled event 'TEST_SCHEDULE' failed. Return code = 12. 04/19/2010 08:56:28 Sending results for scheduled event 'TEST_SCHEDULE'. 04/19/2010 08:56:28 Results sent to server for scheduled event 'TEST_SCHEDULE'. 04/19/2010 08:56:28 ANS1483I Schedule log pruning started. 04/19/2010 08:56:28 ANS1484I Schedule log pruning finished successfully. 04/19/2010 08:56:28 TSM Backup-Archive Client Version 5, Release 5, Level 2.0 04/19/2010 08:56:28 Querying server for next scheduled event. 04/19/2010 08:56:28 Node Name: CLLINUX1 04/19/2010 08:56:28 Session established with server TSMSRV1: Windows 04/19/2010 08:56:28 Server Version 5, Release 5, Level 0.0 04/19/2010 08:56:28 Server date/time: 04/19/2010 08:56:28 Last access: 04/19/2010 08:56:28 04/19/2010 08:56:28 --- SCHEDULEREC QUERY BEGIN 04/19/2010 08:56:28 --- SCHEDULEREC QUERY END 04/19/2010 08:56:28 Next operation scheduled: 04/19/2010 08:56:28 04/19/2010 08:56:28 Schedule Name: TEST_SCHEDULE 04/19/2010 08:56:28 Action:Incremental 04/19/2010 08:56:28 Objects: 04/19/2010 08:56:28 Options: 04/19/2010 08:56:28 Server Window Start: 08:40:00 on 04/20/2010 04/19/2010 08:56:28 04/19/2010 08:56:28 Schedule will be refreshed in 6 hours. This is the info from dmserror.log for this run. 04/19/2010 08:35:34 ANS2820E An interrupt has occurred. The current operation will end and the client will shut down. 04/19/2010 08:36:12 ANS2820E An interrupt has occurred. The current operation will end and the client will shut down. 04/19/2010 08:36:49 ANS2820E An interrupt has occurred. The current operation will end and the client will shut down. 04/19/2010 08:38:24 ANS2820E An interrupt has occurred. The current operation will end and the client will shut down. 04/19/2010 08:56:27 TransErrno: Unexpected error from GetFSInfo:stat, errno = 6 04/19/2010 08:56:27 TransErrno: Unexpected error from GetFSInfo:stat, errno = 6 04/19/2010 08:56:28 TransErrno: Unexpected error from GetFSInfo:stat, errno = 6 04/19/2010 08:56:28 TransErrno: Unexpected error from GetFSInfo:stat, errno = 6 04/19/2010 08:56:28 TransErrno: Unexpected error from lstat, errno = 6 04/19/2010 08:56:28 TransErrno: Unexpected error from GetFSInfo:stat, errno = 6 04/19/2010 08:56:28 TransErrno: Unexpected error from GetFSInfo:stat, errno = 6 04/19/2010 08:56:28 ANS1115W File '/public/
Re: Linux Client failing with ANS1999E error
On Apr 19, 2010, at 6:22 AM, Richard Sims wrote: Jim - That evidence says that the warranted changes have not been made to the client to correct the initially reported problems, which continue. A thorough review of that client should be conducted, starting with 'dsmc query option' and 'dsmc query inclexcl', as the administration of that client are not what they should be. One further thing - which the client administrator should already know: If changes *were* made to the client options file, and its scheduling is *not* being performed via CAD, then obviously the scheduler should have been restarted after client options file changes for them to be in effect. Richard Sims
Re: Linux Client failing with ANS1999E error
Jim - That evidence says that the warranted changes have not been made to the client to correct the initially reported problems, which continue. A thorough review of that client should be conducted, starting with 'dsmc query option' and 'dsmc query inclexcl', as the administration of that client are not what they should be. Richard Sims
Re: Linux Client failing with ANS1999E error
I have attached the dsmsched.log & dsmerror.log Dsmsched.log file 04/17/2010 18:08:35 TSM Backup-Archive Client Version 5, Release 5, Level 2.0 04/17/2010 18:08:35 Querying server for next scheduled event. 04/17/2010 18:08:35 Node Name: CLLINUX1 04/17/2010 18:08:35 Session established with server TSMSRV1: Windows 04/17/2010 18:08:35 Server Version 5, Release 5, Level 0.0 04/17/2010 18:08:35 Server date/time: 04/17/2010 18:08:36 Last access: 04/17/2010 12:08:35 04/17/2010 18:08:35 --- SCHEDULEREC QUERY BEGIN 04/17/2010 18:08:35 --- SCHEDULEREC QUERY END 04/17/2010 18:08:35 Next operation scheduled: 04/17/2010 18:08:35 04/17/2010 18:08:35 Schedule Name: PROD_6PM 04/17/2010 18:08:35 Action:Incremental 04/17/2010 18:08:35 Objects: 04/17/2010 18:08:35 Options: 04/17/2010 18:08:35 Server Window Start: 18:00:00 on 04/17/2010 04/17/2010 18:08:35 04/17/2010 18:08:35 Command will be executed in 13 minutes. 04/17/2010 18:21:35 Executing scheduled command now. 04/17/2010 18:21:35 Node Name: CLLINUX1 04/17/2010 18:21:35 Session established with server TSMSRV1: Windows 04/17/2010 18:21:35 Server Version 5, Release 5, Level 0.0 04/17/2010 18:21:35 Server date/time: 04/17/2010 18:21:36 Last access: 04/17/2010 18:08:36 04/17/2010 18:21:35 --- SCHEDULEREC OBJECT BEGIN PROD_6PM 04/17/2010 18:00:00 04/17/2010 18:21:35 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/17/2010 18:21:35 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/17/2010 18:21:36 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/17/2010 18:21:36 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/17/2010 18:21:36 Incremental backup of volume '/' 04/17/2010 18:21:36 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/17/2010 18:21:36 Incremental backup of volume '/boot' 04/17/2010 18:21:36 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/17/2010 18:21:36 Incremental backup of volume '/public' 04/17/2010 18:21:36 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/17/2010 18:21:36 Incremental backup of volume '/pictometry' 04/17/2010 18:21:36 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/17/2010 18:21:36 Incremental backup of volume '/public/proval-old' 04/17/2010 18:21:37 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/17/2010 18:21:37 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/17/2010 18:21:38 ANS1898I * Processed 9,500 files * 04/17/2010 18:21:39 ANS1898I * Processed54,000 files * 04/17/2010 18:21:40 ANS1898I * Processed85,500 files * 04/17/2010 18:21:41 ANS1898I * Processed 115,500 files * 04/17/2010 18:21:42 ANS1898I * Processed 124,500 files * 04/17/2010 18:21:42 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/17/2010 18:21:42 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/17/2010 18:21:42 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/17/2010 18:21:42 ANS1898I * Processed 167,500 files * 04/17/2010 18:21:42 Successful incremental backup of '/public/proval-old' 04/17/2010 18:21:42 ANS1999E Incremental processing of '/' stopped. 04/17/2010 18:21:42 --- SCHEDULEREC STATUS BEGIN 04/17/2010 18:21:42 Total number of objects inspected: 167,621 04/17/2010 18:21:42 Total number of objects backed up:0 04/17/2010 18:21:42 Total number of objects updated: 0 04/17/2010 18:21:42 Total number of objects rebound: 0 04/17/2010 18:21:42 Total number of objects deleted: 0 04/17/2010 18:21:42 Total number of objects expired: 0 04/17/2010 18:21:42 Total number of objects failed: 0 04/17/2010 18:21:42 Total number of bytes transferred: 0 B 04/17/2010 18:21:42 Data transfer time:0.00 sec 04/17/2010 18:21:42 Network data transfer rate:0.00 KB/sec 04/17/2010 18:21:42 Aggregate data transfer rate: 0.00 KB/sec 04/17/2010 18:21:42 Objects compressed by:0% 04/17/2010 18:21:42 Elapsed processing time: 00:00:07 04/17/2010 18:21:42 --- SCHEDULEREC STATUS END 04/17/2010 18:21:42 ANS1028S An internal program error occurred. 04/17/2010 18:21:42 --- SCHEDULEREC OBJECT END PROD_6PM 04/17/2010 18:00:00 04/17/2010 18:21:42 ANS1512E Scheduled event 'PROD_6PM' failed. Return code = 12. 04/17/2010 18:21:42 Sending results for scheduled event 'PROD_6PM'. 0
Re: Linux Client failing with ANS1999E error
Hi Richard & Len, Thanks for your input. I made the recommended change to the dsm.sys and took out the virtual mount point. I also added an exclude.dir "/public" . This is the latest message. 2010-04-17 18:21 ANR2579E Schedule PROD_6PM in domain PROD for node CLLINUX1 failed (return code 12). (SESSION: 12576) I do not have access to the client schedule log file at this time, but when the admin of the box sends me the file, I will repost. Thanks, -Original Message- From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On Behalf Of Richard Sims Sent: Friday, April 16, 2010 1:22 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: Linux Client failing with ANS1999E error Jim - The backup log doesn't match timewise with the other logs, but I'll assume they are from multiple attempts of the same thing. It looks like the /public/proval-old object is bad in some way, and needs examination. If that system hasn't had fsck's in a while, it may be time. I'd code an Exclude for it (or change the 'Exclude /public/*' to an Exclude.Dir on /public, for at least performance reasons) and reattempt the backup. And remove the Virtualmountpoint for /, as it is a file system. Richard Sims
Re: Linux Client failing with ANS1999E error
I have not seen this error message before. I think that if you remove the following line from the opt file your errors will go away. VIRTUALMOUNTPOINT / len -Original Message- From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On Behalf Of James Choate Sent: Friday, April 16, 2010 2:51 PM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] Linux Client failing with ANS1999E error Has anyone seen this error? I can't figure out whats going on and the messages guide isn't to helpful in resolving this error. Below are the client OS, BA client version, Server version, dsm.sys from the client, error in the dsmsched.log, dsmerror.log and a snippet from the srvr actlog file. Any help is appreciated. I have a linux (Fedora 10) client running Client Version 5, Release 5, Level 2.0 The client is new to this TSM server. The TSM server Windows 2003 Enterprise R2 Server Version 5, Release 5, Level 0.0 My dsm.sys file looks like: VIRTUALMOUNTPOINT / ERRORLOGRETENTION 7 D SCHEDLOGRETENTION 7 D MAXCMDRETRIES 4 QUERYSCHEDPERIOD 1 EXCLUDE "/.../*" EXCLUDE.DIR "/tmp" EXCLUDE.DIR "/var/log" EXCLUDE.FILE "/opt/tivoli/tsm/client/ba/bin/*.log" INCLUDE "/etc/*" INCLUDE "/public/*" INCLUDE "/pictometry/*" INCLUDE "/opt/tivoli/tsm/*" INCLUDE "/root/*" PASSWORDACCESS GENERATE COMMMethod TCPip TCPPort1500 TCPServeraddress MYTSMSRVR nodename CLLINUX1 Running the backup. #dsmc incr IBM Tivoli Storage Manager Command Line Backup/Archive Client Interface Client Version 5, Release 5, Level 2.0 Client date/time: 04/16/2010 12:16:44 (c) Copyright by IBM Corporation and other(s) 1990, 2009. All Rights Reserved. Node Name: CLLINUX1 Session established with server MYTSMSRVR: Windows Server Version 5, Release 5, Level 0.0 Server date/time: 04/16/2010 12:16:44 Last access: 04/16/2010 12:12:44 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. Incremental backup of volume '/' ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. Incremental backup of volume '/boot' ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. Incremental backup of volume '/public' ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. Incremental backup of volume '/pictometry' ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. Incremental backup of volume '/public/proval-old' ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. ANS1898I * Processed 9,500 files * ANS1898I * Processed37,000 files * ANS1898I * Processed74,500 files * ANS1898I * Processed 108,500 files * ANS1898I * Processed 124,500 files * ANS1898I * Processed 165,500 files * ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. ANS1898I * Processed 167,500 files * Successful incremental backup of '/public/proval-old' ANS1999E Incremental processing of '/' stopped. Total number of objects inspected: 167,621 Total number of objects backed up:0 Total number of objects updated: 0 Total number of objects rebound: 0 Total number of objects deleted: 0 Total number of objects expired: 0 Total number of objects failed: 0 Total number of bytes transferred: 0 B Data transfer time:0.00 sec Network data transfer rate:0.00 KB/sec Aggregate data transfer rate: 0.00 KB/sec Objects compressed by:0% Elapsed processing time: 00:00:11 ANS1028S An internal program error occurred. dsmerror.log 04/16/2010 12:05:02 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/16/2010 12:05:03 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/16/2010 12:05:03 TransErrno: Unexpected error from GetFSInfo:stat, errno = 6 04/16/2010 12:05:03 TransErrno
Re: Linux Client failing with ANS1999E error
Jim - The backup log doesn't match timewise with the other logs, but I'll assume they are from multiple attempts of the same thing. It looks like the /public/proval-old object is bad in some way, and needs examination. If that system hasn't had fsck's in a while, it may be time. I'd code an Exclude for it (or change the 'Exclude /public/*' to an Exclude.Dir on /public, for at least performance reasons) and reattempt the backup. And remove the Virtualmountpoint for /, as it is a file system. Richard Sims
Linux Client failing with ANS1999E error
Has anyone seen this error? I can't figure out whats going on and the messages guide isn't to helpful in resolving this error. Below are the client OS, BA client version, Server version, dsm.sys from the client, error in the dsmsched.log, dsmerror.log and a snippet from the srvr actlog file. Any help is appreciated. I have a linux (Fedora 10) client running Client Version 5, Release 5, Level 2.0 The client is new to this TSM server. The TSM server Windows 2003 Enterprise R2 Server Version 5, Release 5, Level 0.0 My dsm.sys file looks like: VIRTUALMOUNTPOINT / ERRORLOGRETENTION 7 D SCHEDLOGRETENTION 7 D MAXCMDRETRIES 4 QUERYSCHEDPERIOD 1 EXCLUDE "/.../*" EXCLUDE.DIR "/tmp" EXCLUDE.DIR "/var/log" EXCLUDE.FILE "/opt/tivoli/tsm/client/ba/bin/*.log" INCLUDE "/etc/*" INCLUDE "/public/*" INCLUDE "/pictometry/*" INCLUDE "/opt/tivoli/tsm/*" INCLUDE "/root/*" PASSWORDACCESS GENERATE COMMMethod TCPip TCPPort1500 TCPServeraddress MYTSMSRVR nodename CLLINUX1 Running the backup. #dsmc incr IBM Tivoli Storage Manager Command Line Backup/Archive Client Interface Client Version 5, Release 5, Level 2.0 Client date/time: 04/16/2010 12:16:44 (c) Copyright by IBM Corporation and other(s) 1990, 2009. All Rights Reserved. Node Name: CLLINUX1 Session established with server MYTSMSRVR: Windows Server Version 5, Release 5, Level 0.0 Server date/time: 04/16/2010 12:16:44 Last access: 04/16/2010 12:12:44 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. Incremental backup of volume '/' ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. Incremental backup of volume '/boot' ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. Incremental backup of volume '/public' ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. Incremental backup of volume '/pictometry' ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. Incremental backup of volume '/public/proval-old' ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. ANS1898I * Processed 9,500 files * ANS1898I * Processed37,000 files * ANS1898I * Processed74,500 files * ANS1898I * Processed 108,500 files * ANS1898I * Processed 124,500 files * ANS1898I * Processed 165,500 files * ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. ANS1898I * Processed 167,500 files * Successful incremental backup of '/public/proval-old' ANS1999E Incremental processing of '/' stopped. Total number of objects inspected: 167,621 Total number of objects backed up:0 Total number of objects updated: 0 Total number of objects rebound: 0 Total number of objects deleted: 0 Total number of objects expired: 0 Total number of objects failed: 0 Total number of bytes transferred: 0 B Data transfer time:0.00 sec Network data transfer rate:0.00 KB/sec Aggregate data transfer rate: 0.00 KB/sec Objects compressed by:0% Elapsed processing time: 00:00:11 ANS1028S An internal program error occurred. dsmerror.log 04/16/2010 12:05:02 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/16/2010 12:05:03 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/16/2010 12:05:03 TransErrno: Unexpected error from GetFSInfo:stat, errno = 6 04/16/2010 12:05:03 TransErrno: Unexpected error from lstat, errno = 6 04/16/2010 12:05:03 ANS1278W Virtual mount point '/' is a file system. It will be backed up as a file system. 04/16/2010 12:05:03 TransErrno: Unexpected error from GetFSInfo:stat, errno = 6 04/16/2010 12:05:04 ANS1999E Incremental processing of '/public/proval-old' stopped. 04/16/2010 12:05:04 ANS1999E Incremental processing of '/' stopped. 04/16/2010 12:05:04 ANS1028S An internal program error occurred. 04/16/2010 12:05:04 ANS1512E Scheduled event 'PROD_12PM' failed. Return code = 12. Q actlog for error message: 04/16/2010 12:00:09 ANR0406I Session 11017 started for node CLLINUX1 (Linux86) (Tcp/Ip CLLINUX1(41275)). (S