Re: [Veritas-bu] Full backup followed by a Differential= Another Full
Thanks for the replies/pointers to this question. The 200644.htm doc was helpful. Now, I've tested 3 environments: Isilon - NDMP; NetApp - NDMP and Solaris - Standard backup. In each case, I used a chgrp command to change the ctime of a small set of files. I then kicked off a Differential for that client. And in each case the small set of files that I had changed were included in the Differential Incremental for each client. I don't have USE_CTIME_FOR_INCREMENTALS in any bp.conf files on clients/media servers or master server. So, I'm still confused, and I guess I'll open up a ticket with Symantec to try and get to the bottom of this. Thanks again, Randy From: Donaldson, Mark mark.donald...@staples.com To: Randy Doering rdoeri...@verizon.net; A Darren Dunham ddun...@taos.com; Veritas-bu@mailman.eng.auburn.edu Sent: Wednesday, December 10, 2008 6:37:52 PM Subject: RE: [Veritas-bu] Full backup followed by a Differential= Another Full Netbackup, by default, uses mtime for incrementals. It can be configured to use C-time in bp.conf but that's not usual at all. More here: http://seer.support.veritas.com/docs/200644.htm -M -Original Message- From: veritas-bu-boun...@mailman.eng.auburn.edu [mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of Randy Doering Sent: Wednesday, December 10, 2008 3:42 PM To: A Darren Dunham; Veritas-bu@mailman.eng.auburn.edu Subject: Re: [Veritas-bu] Full backup followed by a Differential= Another Full Mystery solved - or so it seems. Last week we had a helpdesk ticket where the user wanted some chmod/chgrp modifications of all files/directories in this area. My co-worker did that on Thursday/Friday of last week. Seems this changes the ChangeTime (ctime), which NBU is using for determining if something has Changed. I then did a Differential, and it was looking at the ctime. For one of the files: stat error.txt File: `error.txt' Size: 338713 Blocks: 962 IO Block: 4096 regular file Device: 1dh/29d Inode: 4365470520 Links: 1 Access: (0664/-rw-rw-r--) Uid: ( 2370/ cgoina) Gid: ( 9003/jtc-solexa) Access: 2008-10-15 09:44:41.235029187 -0400 Modify: 2008-10-15 09:44:41.235029187 -0400 Change: 2008-12-09 19:00:21.895757388 -0500 Turns out, the chmod/chgrp needed to be rerun on Monday/Tuesday this week. Is there a way for NBU to look at mtime instead of ctime? Thanks, Randy - Original Message - From: A Darren Dunham ddun...@taos.com To: Veritas-bu@mailman.eng.auburn.edu Sent: Wednesday, December 10, 2008 11:23 AM Subject: Re: [Veritas-bu] Full backup followed by a Differential = Another Full On Wed, Dec 10, 2008 at 07:42:41AM -0800, Randy Doering wrote: Backup Client = Isilon NAS Unit - NDMP. NBU version is 6.5.3. With this?one (10+TBs, 1,643,099 files) during the backup NBU said it was doing a Differential, but when it was all done, it had actually backedup everything again. If I kick off another Differential this coming weekend, I certainly don't want to have yet another backup that gets all of the data once again. Any thoughts? Since this is NDMP, Netbackup isn't making any of the file level decisions. It's just passing a set of parameters to the host (dump level, date of backup, file path, tape drive, etc...) and letting the make the selections. I don't have it in my notes, but you can up the log level of ndmp on the isilon and view exactly what NBU is sending over. It'll show up in the ndmp logs. Ah. Found my notes... On any node of the cluster, edit the file /etc/mcp/templates/syslog.conf and change the line: *.=info /var/log/isi_ndmp_d to: *.* /var/log/isi_ndmp_d This should increase the logging levels on the cluster for ndmp. You may want to check with your support contacts before doing this, but it's just a log level change. With that in place, you can start a manual differential and see what it looks to be doing. You'll be able to kill it so it won't consume tapes for you. If nothing looks obviously wrong, you may have to contact support. Good luck! -- Darren ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
[Veritas-bu] Full backup followed by a Differential = Another Full
Greetings, Backup Client = Isilon NAS Unit - NDMP. NBU version is 6.5.3. Did Complete Fulls, with all of the /ifs/projects/ areas in the Backup Selection, starting on 11/26. This past weekend I kicked off Differential Incrementals for the same Backup Selections. All of the areas in that /ifs/projects appeared to have completed as expected except for one. With this one (10+TBs, 1,643,099 files) during the backup NBU said it was doing a Differential, but when it was all done, it had actually backedup everything again. I can go into Backup and Restore for this client, and choosing show all backups, it will show the Full that was started on 11/26 and will also show the Differential that was started on 12/6 with the same Modification date (prior to 11/26) for each. Wouldn't have noticed so much, but with it taking 4+ days and using 16 tapes, has got me wondering about it. If I kick off another Differential this coming weekend, I certainly don't want to have yet another backup that gets all of the data once again. Any thoughts? Thanks, Randy___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Full backup followed by a Differential = Another Full
You have to setup Cumulative incremental I believe as ndmp backup levels go to 9. After 9 every backup is a full backup unless reset. Cumulative incremental resets it so you'll get a differential next. This is the case on NetApps. -- Rajmund Siwik Broadcom Corporation From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Randy Doering Sent: Wednesday, December 10, 2008 7:43 AM To: Veritas-bu@mailman.eng.auburn.edu Subject: [Veritas-bu] Full backup followed by a Differential = Another Full Greetings, Backup Client = Isilon NAS Unit - NDMP. NBU version is 6.5.3. Did Complete Fulls, with all of the /ifs/projects/ areas in the Backup Selection, starting on 11/26. This past weekend I kicked off Differential Incrementals for the same Backup Selections. All of the areas in that /ifs/projects appeared to have completed as expected except for one. With this one (10+TBs, 1,643,099 files) during the backup NBU said it was doing a Differential, but when it was all done, it had actually backedup everything again. I can go into Backup and Restore for this client, and choosing show all backups, it will show the Full that was started on 11/26 and will also show the Differential that was started on 12/6 with the same Modification date (prior to 11/26) for each. Wouldn't have noticed so much, but with it taking 4+ days and using 16 tapes, has got me wondering about it. If I kick off another Differential this coming weekend, I certainly don't want to have yet another backup that gets all of the data once again. Any thoughts? Thanks, Randy ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Full backup followed by a Differential = Another Full
Mystery solved - or so it seems. Last week we had a helpdesk ticket where the user wanted some chmod/chgrp modifications of all files/directories in this area. My co-worker did that on Thursday/Friday of last week. Seems this changes the ChangeTime (ctime), which NBU is using for determining if something has Changed. I then did a Differential, and it was looking at the ctime. For one of the files: stat error.txt File: `error.txt' Size: 338713 Blocks: 962IO Block: 4096 regular file Device: 1dh/29d Inode: 4365470520 Links: 1 Access: (0664/-rw-rw-r--) Uid: ( 2370/ cgoina) Gid: ( 9003/jtc-solexa) Access: 2008-10-15 09:44:41.235029187 -0400 Modify: 2008-10-15 09:44:41.235029187 -0400 Change: 2008-12-09 19:00:21.895757388 -0500 Turns out, the chmod/chgrp needed to be rerun on Monday/Tuesday this week. Is there a way for NBU to look at mtime instead of ctime? Thanks, Randy - Original Message - From: A Darren Dunham [EMAIL PROTECTED] To: Veritas-bu@mailman.eng.auburn.edu Sent: Wednesday, December 10, 2008 11:23 AM Subject: Re: [Veritas-bu] Full backup followed by a Differential = Another Full On Wed, Dec 10, 2008 at 07:42:41AM -0800, Randy Doering wrote: Backup Client = Isilon NAS Unit - NDMP. NBU version is 6.5.3. With this?one (10+TBs, 1,643,099 files) during the backup NBU said it was doing a Differential, but when it was all done, it had actually backedup everything again. If I kick off another Differential this coming weekend, I certainly don't want to have yet another backup that gets all of the data once again. Any thoughts? Since this is NDMP, Netbackup isn't making any of the file level decisions. It's just passing a set of parameters to the host (dump level, date of backup, file path, tape drive, etc...) and letting the make the selections. I don't have it in my notes, but you can up the log level of ndmp on the isilon and view exactly what NBU is sending over. It'll show up in the ndmp logs. Ah. Found my notes... On any node of the cluster, edit the file /etc/mcp/templates/syslog.conf and change the line: *.=info /var/log/isi_ndmp_d to: *.* /var/log/isi_ndmp_d This should increase the logging levels on the cluster for ndmp. You may want to check with your support contacts before doing this, but it's just a log level change. With that in place, you can start a manual differential and see what it looks to be doing. You'll be able to kill it so it won't consume tapes for you. If nothing looks obviously wrong, you may have to contact support. Good luck! -- Darren ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Full backup followed by a Differential = Another Full
On Wed, Dec 10, 2008 at 05:41:30PM -0500, Randy Doering wrote: Turns out, the chmod/chgrp needed to be rerun on Monday/Tuesday this week. Is there a way for NBU to look at mtime instead of ctime? Again, this is NDMP. NBU isn't looking at anything. It's telling the NDMP host please take a diff backup of everything in this path since this point in time. What the NDMP host does with that information (how it interprets what has changed) is beyond the direct control of NBU or any backup software. However, it is possible that the behavior is controllable within the NDMP host. But that's up to Isilon, not NBU. Either check the documentation or open a ticket with them and see if this is possible. NBU can pass in environment variables to NDMP. As an example, you can tell the NDMP server to use TAR or DUMP as the file format. -- Darren ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Full backup followed by a Differential= Another Full
Netbackup, by default, uses mtime for incrementals. It can be configured to use C-time in bp.conf but that's not usual at all. More here: http://seer.support.veritas.com/docs/200644.htm -M -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Randy Doering Sent: Wednesday, December 10, 2008 3:42 PM To: A Darren Dunham; Veritas-bu@mailman.eng.auburn.edu Subject: Re: [Veritas-bu] Full backup followed by a Differential= Another Full Mystery solved - or so it seems. Last week we had a helpdesk ticket where the user wanted some chmod/chgrp modifications of all files/directories in this area. My co-worker did that on Thursday/Friday of last week. Seems this changes the ChangeTime (ctime), which NBU is using for determining if something has Changed. I then did a Differential, and it was looking at the ctime. For one of the files: stat error.txt File: `error.txt' Size: 338713 Blocks: 962IO Block: 4096 regular file Device: 1dh/29d Inode: 4365470520 Links: 1 Access: (0664/-rw-rw-r--) Uid: ( 2370/ cgoina) Gid: ( 9003/jtc-solexa) Access: 2008-10-15 09:44:41.235029187 -0400 Modify: 2008-10-15 09:44:41.235029187 -0400 Change: 2008-12-09 19:00:21.895757388 -0500 Turns out, the chmod/chgrp needed to be rerun on Monday/Tuesday this week. Is there a way for NBU to look at mtime instead of ctime? Thanks, Randy - Original Message - From: A Darren Dunham [EMAIL PROTECTED] To: Veritas-bu@mailman.eng.auburn.edu Sent: Wednesday, December 10, 2008 11:23 AM Subject: Re: [Veritas-bu] Full backup followed by a Differential = Another Full On Wed, Dec 10, 2008 at 07:42:41AM -0800, Randy Doering wrote: Backup Client = Isilon NAS Unit - NDMP. NBU version is 6.5.3. With this?one (10+TBs, 1,643,099 files) during the backup NBU said it was doing a Differential, but when it was all done, it had actually backedup everything again. If I kick off another Differential this coming weekend, I certainly don't want to have yet another backup that gets all of the data once again. Any thoughts? Since this is NDMP, Netbackup isn't making any of the file level decisions. It's just passing a set of parameters to the host (dump level, date of backup, file path, tape drive, etc...) and letting the make the selections. I don't have it in my notes, but you can up the log level of ndmp on the isilon and view exactly what NBU is sending over. It'll show up in the ndmp logs. Ah. Found my notes... On any node of the cluster, edit the file /etc/mcp/templates/syslog.conf and change the line: *.=info /var/log/isi_ndmp_d to: *.* /var/log/isi_ndmp_d This should increase the logging levels on the cluster for ndmp. You may want to check with your support contacts before doing this, but it's just a log level change. With that in place, you can start a manual differential and see what it looks to be doing. You'll be able to kill it so it won't consume tapes for you. If nothing looks obviously wrong, you may have to contact support. Good luck! -- Darren ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
