This is from a co-worker FWIW, the TSM server is 5.2.3.2 on AIX 5.1. No hardware or network changes !
***************************************************************************** The incremental backup for this server last worked correctly on October 23. The October 24 incremental failed with this error: ANS1225E Insufficient memory for file compression/expansion Shortly thereafter, when the client attempted to get the next schedule, I started getting these errors: ANS1029E Communications have been dropped. Will attempt to get schedule from server again in 20 minutes. I restarted the client on October 25, and that night's incremental again got this error: ANS1225E Insufficient memory for file compression/expansion This was followed by the same scenario of "Communications have been dropped" errors. I then turned off compression and restarted the client. The October 26 incremental failed with this error: ANS1030E System ran out of memory. Process ended. The filesystems on this server haven't changed significantly since the backup last worked. The incremental normally inspects about 13,000,000 entries and backs up about 250,000 of them. It's an AIX 4.3.3 system, the TSM client is at 5.1.5.15 and the filesystems are JFS. The server has 2GB of memory with 3GB of paging space. There are no relevant entries in the AIX error log. Looking at the dsmerror.log, I'm wondering what the heck is "ERROR: **llStrP == NULL in InsertSlashHack!"? And the only thing I see out of the ordinary related to this message: PrivIncrFileSpace: Received rc=102 from fioGetDirEntries: /var/spool/imap2 /titan/oa4/user/pandurangiaa is a directory named "/var/spool/imap2/titan/oa4/user/pandurangiaa/Abhi Panderguy" (note the space in the last part of the name), but that directory dates back to August 5. Any ideas? DSMSCHED.LOG: Executing scheduled command now. 10/26/04 18:30:08 --- SCHEDULEREC OBJECT BEGIN MAIL.DAILY 10/26/04 18:30:00 10/26/04 19:51:00 ANS1228E Sending of object '/var/spool/imap2/aa4/user/aabaski/25074.' failed 10/26/04 19:51:01 ANS4005E Error processing '/var/spool/imap2/aa4/user/aabaski/25074.': file not found 10/26/04 20:48:35 ANS1228E Sending of object '/var/spool/imap2/co5/user/crisinatita/571.' failed 10/26/04 20:48:35 ANS4005E Error processing '/var/spool/imap2/co5/user/crisinatita/571.': file not found 10/26/04 20:48:57 ANS1228E Sending of object '/var/spool/imap2/co5/user/crockermd2/126.' failed 10/26/04 20:48:57 ANS4005E Error processing '/var/spool/imap2/co5/user/crockermd2/126.': file not found 10/26/04 20:49:10 ANS1228E Sending of object '/var/spool/imap2/co5/user/crockermd2/128.' failed 10/26/04 20:49:10 ANS4005E Error processing '/var/spool/imap2/co5/user/crockermd2/128.': file not found 10/26/04 21:16:09 ANS1228E Sending of object '/var/spool/imap2/ha4/user/harrisjl4/2460.' failed 10/26/04 21:16:09 ANS4005E Error processing '/var/spool/imap2/ha4/user/harrisjl4/2460.': file not found 10/26/04 21:27:51 ANS1228E Sending of object '/var/spool/imap2/mo4/user/montisanomj/613.' failed 10/26/04 21:27:52 ANS4005E Error processing '/var/spool/imap2/mo4/user/montisanomj/613.': file not found 10/26/04 21:27:57 ANS1228E Sending of object '/var/spool/imap2/mo4/user/montisanomj/614.' failed 10/26/04 21:27:57 ANS4005E Error processing '/var/spool/imap2/mo4/user/montisanomj/614.': file not found 10/26/04 22:25:31 ANS1802E Incremental backup of '/var/spool/imap2' finished with 7 failure 10/26/04 22:30:08 --- SCHEDULEREC STATUS BEGIN 10/26/04 22:30:08 Total number of objects inspected: 3,222,540 10/26/04 22:30:08 Total number of objects backed up: 143,381 10/26/04 22:30:08 Total number of objects updated: 0 10/26/04 22:30:08 Total number of objects rebound: 0 10/26/04 22:30:08 Total number of objects deleted: 0 10/26/04 22:30:08 Total number of objects expired: 28,277 10/26/04 22:30:08 Total number of objects failed: 7 10/26/04 22:30:08 Total number of bytes transferred: 4.47 GB 10/26/04 22:30:08 Data transfer time: 477.21 sec 10/26/04 22:30:08 Network data transfer rate: 9,832.12 KB/sec 10/26/04 22:30:08 Aggregate data transfer rate: 325.85 KB/sec 10/26/04 22:30:08 Objects compressed by: 0% 10/26/04 22:30:08 Elapsed processing time: 03:59:59 10/26/04 22:30:08 --- SCHEDULEREC STATUS END 10/26/04 22:30:08 ANS1030E System ran out of memory. Process ended. 10/26/04 22:30:08 --- SCHEDULEREC OBJECT END MAIL.DAILY 10/26/04 18:30:00 10/26/04 22:30:08 ANS1512E Scheduled event 'MAIL.DAILY' failed. Return code = 12. DSMERROR.LOG: 10/26/04 21:51:06 ERROR: **llStrP == NULL in InsertSlashHack! <snippage of duplicate messages> 10/26/04 21:51:07 ERROR: **llStrP == NULL in InsertSlashHack! 10/26/04 21:51:12 Thread creation failed; rc=11. 10/26/04 21:51:13 ERROR: **llStrP == NULL in InsertSlashHack! <snippage of duplicate messages> 10/26/04 21:51:19 ERROR: **llStrP == NULL in InsertSlashHack! 10/26/04 22:22:23 PrivIncrFileSpace: Received rc=102 from fioGetDirEntries: /var/spool/imap2 /titan/oa4/user/pandurangiaa 10/26/04 22:25:31 ANS1802E Incremental backup of '/var/spool/imap2' finished with 7 failure 10/26/04 22:30:08 ANS1030E System ran out of memory. Process ended. 10/26/04 22:30:08 ANS1512E Scheduled event 'MAIL.DAILY' failed. Return code = 12.