Re: TDP for SQL Server Transaction Log Restores
Paul, I don't think this would be possible. It would require the application to understand the object naming as well as other "internals" on how it stores the data. Typically, in the case of the Data Protection clients (or any API application really), the application that backs up the data is really the only one that can restore it AND make use of it. Many times, that is because it must interact with the application in order to make sense of the data. The only thing I could see is if the Entegra tool "drove" the TDP for SQL command-line interface... but then again I don't think that is what you are looking for. What would Entegra do with the stream of data anyway? It is a proprietary stream of data only known by Microsoft SQL Server. You should investigate what are the exact requirements of the Entegra tool and how it is supposed to "hook" in to SQL Server. Thanks, Del Del Hoobler IBM Corporation [EMAIL PROTECTED] - Never cut what can be untied. - Commit yourself to constant improvement. === > I guess I misunderstood what the product is probably doing. The company > that has this auditing product is LUMIGENT, www.lumigent.com. The product > is Entegra. I have not talked to their developers as yet, but they seem to > be interested in retrieving the objects from TSM through the API. > > We have just implemented TDPSQLC for logs and it works great. This wrinkle > though is driving us back to using log intermediate files as you have > described and backing them up. > > Would it be feasible for a tool like Entegra to read the data back throught > the API, simulating the TDP for SQL Server?
Re: TDP for SQL Server Transaction Log Restores
OK, I see what you mean. I guess I misunderstood what the product is probably doing. The company that has this auditing product is LUMIGENT, www.lumigent.com. The product is Entegra. I have not talked to their developers as yet, but they seem to be interested in retrieving the objects from TSM through the API. We have just implemented TDPSQLC for logs and it works great. This wrinkle though is driving us back to using log intermediate files as you have described and backing them up. Would it be feasible for a tool like Entegra to read the data back throught the API, simulating the TDP for SQL Server? Paul D. Seay, Jr. Technical Specialist Naptheon Inc. 757-688-8180 -Original Message- From: Del Hoobler [mailto:[EMAIL PROTECTED]] Sent: Wednesday, December 04, 2002 10:49 AM To: [EMAIL PROTECTED] Subject: Re: TDP for SQL Server Transaction Log Restores Paul, I am not sure what you mean exactly. The SQL "logging" process does not have separate "log" segment files. The "log" that TDP for SQL backs up is just a "stream" of data that only SQL knows what to do with. It is not like a log file for Exchange or Domino. There is no way to just restore a "log" to a "file" with DP for SQL. You can only restore it to a live SQL server. The SQL server controls where that stream of data is written to. You might want to take a closer look at the requirement to find out how they are asking that this be done and in what fashion. Thanks, Del Del Hoobler IBM Corporation [EMAIL PROTECTED] - Never cut what can be untied. - Commit yourself to constant improvement. > We have a requirement to restore the transaction logs back to their > disk location either on the orginal or an alternate server or to be > able to post > process the transaction logs as disk files some kind of way before > they get > deleted after a TDP for SQL Server Backup. > > My DBAs tell me SQL server deletes the transaction logs immediately > after a > backup which presents a problem for a third party utility we want to > do auditing with.
Re: TDP for SQL Server Transaction Log Restores
Need clarifcation of your statements. Let me explain. TDP SQL does not only do FULL backups of the SQL DB's, but can also do LOG backups - live and it will not put these logs on disk, it backs them up to TSM server like the FULL backups. Now when the LOG backup happens, it truncates the SQL transaction log for the DB's backed up on completion of the LOG backup. Now if the LOG backups are done by DBA's through SQL only and these logs files are put on disk somewhere for a regular TSM B/A client to pick up, then that is a different story. You need to know how often, naming convention etc. that the DBA's use so you can adequately back these up. They may delete these by their own method after a certain time, say after the next FULL backup. This requires more planning on your and DBA's part to work correctly. We use both LOG backup methods here with a number of variations for various reasons. In some cases we do hourly LOG backups We have done successful restores of DB and LOGs for both situations. David B. Longo System Administrator Health First, Inc. 3300 Fiske Blvd. Rockledge, FL 32955-4305 PH 321.434.5536 Pager 321.634.8230 Fax:321.434.5509 [EMAIL PROTECTED] >>> [EMAIL PROTECTED] 12/04/02 09:17AM >>> We have a requirement to restore the transaction logs back to their disk location either on the orginal or an alternate server or to be able to post process the transaction logs as disk files some kind of way before they get deleted after a TDP for SQL Server Backup. My DBAs tell me SQL server deletes the transaction logs immediately after a backup which presents a problem for a third party utility we want to do auditing with. Any ideas? Paul D. Seay, Jr. Technical Specialist Naptheon Inc. 757-688-8180 "MMS " made the following annotations on 12/04/2002 11:43:06 AM -- This message is for the named person's use only. It may contain confidential, proprietary, or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it, and notify the sender. You must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Health First reserves the right to monitor all e-mail communications through its networks. Any views or opinions expressed in this message are solely those of the individual sender, except (1) where the message states such views or opinions are on behalf of a particular entity; and (2) the sender is authorized by the entity to give such views or opinions. ==
Re: TDP for SQL Server Transaction Log Restores
Paul, I am not sure what you mean exactly. The SQL "logging" process does not have separate "log" segment files. The "log" that TDP for SQL backs up is just a "stream" of data that only SQL knows what to do with. It is not like a log file for Exchange or Domino. There is no way to just restore a "log" to a "file" with DP for SQL. You can only restore it to a live SQL server. The SQL server controls where that stream of data is written to. You might want to take a closer look at the requirement to find out how they are asking that this be done and in what fashion. Thanks, Del Del Hoobler IBM Corporation [EMAIL PROTECTED] - Never cut what can be untied. - Commit yourself to constant improvement. > We have a requirement to restore the transaction logs back to their disk > location either on the orginal or an alternate server or to be able to post > process the transaction logs as disk files some kind of way before they get > deleted after a TDP for SQL Server Backup. > > My DBAs tell me SQL server deletes the transaction logs immediately after a > backup which presents a problem for a third party utility we want to do > auditing with.
Re: TDP for SQL Server Transaction Log Restores
TDP docs say to set verd=reto=0. Just change them and go down the "restore
deleted file" road.
Zlatko Krastev
IT Consultant
"Seay, Paul" <[EMAIL PROTECTED]>
Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
04.12.2002 16:17
Please respond to "ADSM: Dist Stor Manager"
To: [EMAIL PROTECTED]
cc:
Subject:TDP for SQL Server Transaction Log Restores
We have a requirement to restore the transaction logs back to their disk
location either on the orginal or an alternate server or to be able to
post
process the transaction logs as disk files some kind of way before they
get
deleted after a TDP for SQL Server Backup.
My DBAs tell me SQL server deletes the transaction logs immediately after
a
backup which presents a problem for a third party utility we want to do
auditing with.
Any ideas?
Paul D. Seay, Jr.
Technical Specialist
Naptheon Inc.
757-688-8180
TDP for SQL Server Transaction Log Restores
We have a requirement to restore the transaction logs back to their disk location either on the orginal or an alternate server or to be able to post process the transaction logs as disk files some kind of way before they get deleted after a TDP for SQL Server Backup. My DBAs tell me SQL server deletes the transaction logs immediately after a backup which presents a problem for a third party utility we want to do auditing with. Any ideas? Paul D. Seay, Jr. Technical Specialist Naptheon Inc. 757-688-8180
Re: Transaction Log Restores
> With this specific restore...around 200 logs were applied before it > required the logs I had on disk...so as soon as I started the log > restoreI then copied the logs into the specified restore folder. By > the time the restore needed the logs that were on disk, everything had > already been copied across. I noticed also when these logs were being > applied that they didn't show up on the GUI, I guess this is because they > weren't archived As I said before they get removed from the restore > folder once they are applied. Once it has finsihed with these logs it > continues applying the logs from tape. > > I haven't tried copying the log files across only once they are being > called for, I'm not sure this would work as the process doesn't wait for > that long before deciding the logs aren't available so there isn't a lot > of time to copy them across. Marc, Richard, It is good that you had enough time to copy those logs in after you started the ACTIVATE but before the Domino server (recovery manager) needed them, and thus, the Domino server saw them in the log directory and used them. And so, the Domino Server did not need to call the TDP for Domino code to restore them. It would have been nice to be able to copy those logs into the directory before the ACTIVATE... but for some reason the Domino Server does not like that. It seems that this was an isolated case of manual intervention causing turmoil. You may want to increase the frequency of the log backups or add some additional disk space to your log device so that a log full condition does not happen on the Domino server. TDP for Domino "log restore" (during activation with applylogs) simply responds to the requests from the Domino server if the Domino Server recovery manager needs a log that is not present on the local log disk. That is, it is a "callback" mechanism initiated by the Domino Server recovery manager. By the way, TDP for Domino, by default, has PREFETCH turned on... ... which means that it will always try to stay one log ahead of what the Domino Server is asking for. It does this on a different thread so that it can maximize performance during activation. Thanks, Del Del Hoobler IBM Corporation [EMAIL PROTECTED] - Never cut what can be untied. - Commit yourself to constant improvement.
Re: Transaction Log Restores
With this specific restore...around 200 logs were applied before it
required the logs I had on disk...so as soon as I started the log
restoreI then copied the logs into the specified restore folder. By
the time the restore needed the logs that were on disk, everything had
already been copied across. I noticed also when these logs were being
applied that they didn't show up on the GUI, I guess this is because they
weren't archived As I said before they get removed from the restore
folder once they are applied. Once it has finsihed with these logs it
continues applying the logs from tape.
I haven't tried copying the log files across only once they are being
called for, I'm not sure this would work as the process doesn't wait for
that long before deciding the logs aren't available so there isn't a lot
of time to copy them across.
Marc.
"Richard Foster" <[EMAIL PROTECTED]>
Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
23-Oct-2002 14:27
Please respond to "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
To: ADSM-L
cc:
Subject:Re: Transaction Log Restores
Marc
Sorry to hammer this home, but ...
> I didn't do anything differently except copy the log files
> after the applylogs command was issued.
I'm guessing that what you're saying is that
- you issue the applylogs command
- the command waits until you copy a log file into the folder
- it applies the log and then deletes it.
You then repeat the copy for each log until you're done.
Is that a correct guess?
What happens if the applylogs needs logs that *have* been archived by the
TDP, either before or after the manual logs? Is the TDP clever enough to
handle that? At a guess, it should
- restore/apply/delete the logs as usuaul, from before the manual set
- wait for you to copy the logs in, as described above
- then continue with the usual restore/apply/delete of logs from after the
manual set
Is it really that clever?
Richard
Re: Transaction Log Restores
Marc Sorry to hammer this home, but ... > I didn't do anything differently except copy the log files > after the applylogs command was issued. I'm guessing that what you're saying is that - you issue the applylogs command - the command waits until you copy a log file into the folder - it applies the log and then deletes it. You then repeat the copy for each log until you're done. Is that a correct guess? What happens if the applylogs needs logs that *have* been archived by the TDP, either before or after the manual logs? Is the TDP clever enough to handle that? At a guess, it should - restore/apply/delete the logs as usuaul, from before the manual set - wait for you to copy the logs in, as described above - then continue with the usual restore/apply/delete of logs from after the manual set Is it really that clever? Richard
Re: Transaction Log Restores
The API appeared to pick these logs up from the restore folder
automatically, I didn't do anything differently except copy the log files
after the applylogs command was issued.
The log files were read and then deleted from the restore folder after
they were applied...one by one. So it is important that the files were
copied into the folder and not just moved from their original location.
Marc.
"Richard Foster" <[EMAIL PROTECTED]>
Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
23-Oct-2002 12:11
Please respond to "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
To: ADSM-L
cc:
Subject:Re: Transaction Log Restores
Marc
> I've found that this is possible to restore a database with the
> log files placed back in the restore area as you say.
> I did also find that I had to copy these log extents into the
> folder after issuing the applylogs command, a NOTES.RIP file
> was created if the log files were already there.
Sorry, I'm a bit confused. What exactly did you do? If you can only copy
the logs into the folder *after* the applylogs command, how do you apply
these logs to the database?
Richard
Re: Transaction Log Restores
Marc > I've found that this is possible to restore a database with the > log files placed back in the restore area as you say. > I did also find that I had to copy these log extents into the > folder after issuing the applylogs command, a NOTES.RIP file > was created if the log files were already there. Sorry, I'm a bit confused. What exactly did you do? If you can only copy the logs into the folder *after* the applylogs command, how do you apply these logs to the database? Richard
Re: Transaction Log Restores
I've found that this is possible to restore a database with the log files
placed back in the restore area as you say. I did also find that I had to
copy these log extents into the folder after issuing the applylogs
command, a NOTES.RIP file was created if the log files were already there.
Thanks.
"Del Hoobler" <[EMAIL PROTECTED]>
Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
22-Oct-2002 13:18
Please respond to "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
To: ADSM-L
cc:
Subject:Re: Transaction Log Restores
>>> Question is, can I still archive these log files so they can be
>>> applied to the previous backup?
>> No. You cannot. That is because Domino no longer "tracks" them.
> Would it be possible to manually copy the 'missing' log files back to
the
> restore area before doing the restore? In that way, maybe the client
would
> find the logs at the time when they were needed in the process?
>
> Alternatively, could you do a restore to a PIT just before the missing
> logs, without an activate, then in some way apply the missing logs, and
> return to the TDP to apply logs and activate after the missing ones?
Richard,
You first idea might work and is the one you should try.
As for your second idea... as you have described,
I really don't know how you could accomplish it.
How do you propose to apply the missing logs?
The only two ways I know to apply logs is through
TDP for Domino and through a Domino server "recovery".
Del
Re: Transaction Log Restores
>>> Question is, can I still archive these log files so they can be >>> applied to the previous backup? >> No. You cannot. That is because Domino no longer "tracks" them. > Would it be possible to manually copy the 'missing' log files back to the > restore area before doing the restore? In that way, maybe the client would > find the logs at the time when they were needed in the process? > > Alternatively, could you do a restore to a PIT just before the missing > logs, without an activate, then in some way apply the missing logs, and > return to the TDP to apply logs and activate after the missing ones? Richard, You first idea might work and is the one you should try. As for your second idea... as you have described, I really don't know how you could accomplish it. How do you propose to apply the missing logs? The only two ways I know to apply logs is through TDP for Domino and through a Domino server "recovery". Del
Re: Transaction Log Restores
Del, >> Question is, can I still archive these log files so they can be >> applied to the previous backup? > No. You cannot. That is because Domino no longer "tracks" them. Would it be possible to manually copy the 'missing' log files back to the restore area before doing the restore? In that way, maybe the client would find the logs at the time when they were needed in the process? Alternatively, could you do a restore to a PIT just before the missing logs, without an activate, then in some way apply the missing logs, and return to the TDP to apply logs and activate after the missing ones? Just a thought. Regards, Richard Foster
Re: Transaction Log Restores
> I have a problem with a TDP Domino restore, we have a hole in our > transaction logs from some that were copied off the disk at a time when > the log disk was full. At the time I thought I'd taken an archive before > the logs were removed (positive in fact) but now when I look the logs > aren't shown. We have taken a full backup of the client since and still > have the logs that were copied to another disk. > Question is, can I still archive these log files so they can be applied to > the previous backup? Marc, No. You cannot. That is because Domino no longer "tracks" them. The Domino Server tells TDP for Domino which logs to archive. So, if it is asking for logs AFTER those "copied off" logs, then it will not go back and ask for the older logs. If the Domino server is not asking for those logs to be archived, then you probably archived them already... or... the Domino Server may have gotten confused and out of sync when you copied them off the device and it could not find them. You can take a look at the TSM Server activity log to see if they were ever archived. TDP for Domino will log each transaction log that it backs up. That way you can tell if it was backed up. Did you also run an "INACTIVE" query to see if they have been forced to inactive mode by the INACTIVATELOGS command? Thanks, Del Del Hoobler IBM Corporation [EMAIL PROTECTED] Thanks, Del Del Hoobler IBM Corporation [EMAIL PROTECTED] - Never cut what can be untied. - Commit yourself to constant improvement.
