[Veritas-bu] NetBackup 7.1 media server backup issue

2011-05-05 Thread Jim . Canon

Hi All,

We're having a strange problem when the media server (AIX 6.1) is backing
up itself. This problem only occurs when the media server attempts to back
up itself. The media server can back up other clients just fine. Other
media servers can back up this media server just fine.

Here is the general output of the backup:

05/05/2011 09:37:01 - begin writing
05/05/2011 09:52:16 - Error bpbrm (pid=7405600) socket read failed: errno =
119 - System call timed out
05/05/2011 09:52:18 - Error bptm (pid=8912950) media manager terminated by
parent process
05/05/2011 09:52:27 - Info bpbkar (pid=9437276) done. status: 13: file read
failed

This isn't necessarily helpful, so I've enabled verbose logging level 5 on
both bpbkar and bpbrm. I'm not seeing any red flags.

I have tried disabling shared memory, reducing the buffers to bare
minimums, excluding what I thought were problem files, but it seems we are
chasing a ghost. It is as if bpbkar stops sending data. The backup runs for
a while, stops on a random file, and sits there until the job times out.

Any ideas?

--
Jim Canon
Sr. Data Protection Analyst
Global Technology Services - AHS
Mylan
781 Chestnut Ridge Road
Morgantown, WV 26505

jim.ca...@mylan.com

Phone: 800.826.9526 ext. 6236
Direct: 304.554.6236
Fax: 304.285.6404
Mobile: 304.224.6815


==
CONFIDENTIALITY NOTICE:  This e-mail message and all attachments transmitted 
with it may contain legally privileged, proprietary and/or confidential 
information intended solely for the use of the addressee.  If you are not the 
intended recipient, you are hereby notified that any review, dissemination, 
distribution, duplication or other use of this message and/or its attachments 
is strictly prohibited.  If you are not the intended recipient, please contact 
the sender by reply e-mail and destroy all copies of the original message and 
its attachments.  Thank you.
==

___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu


Re: [Veritas-bu] NetBackup 7.1 media server backup issue

2011-05-05 Thread Girish Jorapurkar
If I am understanding it correctly, an error occurred while reading a file.

> 05/05/2011 09:52:27 - Info bpbkar (pid=9437276) done.
> status: 13: file read failed

That is strange. See if there is any storage (volume manager or disk I/O) 
related error/fault reported at the time of bpbkar failure.

Also, you can try bpbkar null test - just to ensure that data can be read 
without any random read failures:

bpbkar -nocont  > /dev/null

Hope that helps,
/Girish

--- On Fri, 5/6/11, jim.ca...@mylan.com  wrote:

> From: jim.ca...@mylan.com 
> Subject: [Veritas-bu] NetBackup 7.1 media server backup issue
> To: veritas-bu@mailman.eng.auburn.edu
> Date: Friday, May 6, 2011, 6:45 AM
> 
> Hi All,
> 
> We're having a strange problem when the media server (AIX
> 6.1) is backing
> up itself. This problem only occurs when the media server
> attempts to back
> up itself. The media server can back up other clients just
> fine. Other
> media servers can back up this media server just fine.
> 
> Here is the general output of the backup:
> 
> 05/05/2011 09:37:01 - begin writing
> 05/05/2011 09:52:16 - Error bpbrm (pid=7405600) socket read
> failed: errno =
> 119 - System call timed out
> 05/05/2011 09:52:18 - Error bptm (pid=8912950) media
> manager terminated by
> parent process
> 05/05/2011 09:52:27 - Info bpbkar (pid=9437276) done.
> status: 13: file read
> failed
> 
> This isn't necessarily helpful, so I've enabled verbose
> logging level 5 on
> both bpbkar and bpbrm. I'm not seeing any red flags.
> 
> I have tried disabling shared memory, reducing the buffers
> to bare
> minimums, excluding what I thought were problem files, but
> it seems we are
> chasing a ghost. It is as if bpbkar stops sending data. The
> backup runs for
> a while, stops on a random file, and sits there until the
> job times out.
> 
> Any ideas?
> 
> --
> Jim Canon
> Sr. Data Protection Analyst
> Global Technology Services - AHS
> Mylan
> 781 Chestnut Ridge Road
> Morgantown, WV 26505
> 
> jim.ca...@mylan.com
> 
> Phone: 800.826.9526 ext. 6236
> Direct: 304.554.6236
> Fax: 304.285.6404
> Mobile: 304.224.6815
> 
> 
> ==
> CONFIDENTIALITY NOTICE:  This e-mail message and all
> attachments transmitted with it may contain legally
> privileged, proprietary and/or confidential information
> intended solely for the use of the addressee.  If you
> are not the intended recipient, you are hereby notified that
> any review, dissemination, distribution, duplication or
> other use of this message and/or its attachments is strictly
> prohibited.  If you are not the intended recipient,
> please contact the sender by reply e-mail and destroy all
> copies of the original message and its attachments. 
> Thank you.
> ==
> 
> ___
> Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
> 
___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu


Re: [Veritas-bu] NetBackup 7.1 media server backup issue

2011-05-05 Thread Saran Brar
I have few questions:

- What is in the backup selection.
- Follow NFS and cross mount points selected
- Try splitting up the backup selection in multiple streams and pin point
exactly which file or folder is creating problem.

On Fri, May 6, 2011 at 9:15 AM,  wrote:

>
> Hi All,
>
> We're having a strange problem when the media server (AIX 6.1) is backing
> up itself. This problem only occurs when the media server attempts to back
> up itself. The media server can back up other clients just fine. Other
> media servers can back up this media server just fine.
>
> Here is the general output of the backup:
>
> 05/05/2011 09:37:01 - begin writing
> 05/05/2011 09:52:16 - Error bpbrm (pid=7405600) socket read failed: errno =
> 119 - System call timed out
> 05/05/2011 09:52:18 - Error bptm (pid=8912950) media manager terminated by
> parent process
> 05/05/2011 09:52:27 - Info bpbkar (pid=9437276) done. status: 13: file read
> failed
>
> This isn't necessarily helpful, so I've enabled verbose logging level 5 on
> both bpbkar and bpbrm. I'm not seeing any red flags.
>
> I have tried disabling shared memory, reducing the buffers to bare
> minimums, excluding what I thought were problem files, but it seems we are
> chasing a ghost. It is as if bpbkar stops sending data. The backup runs for
> a while, stops on a random file, and sits there until the job times out.
>
> Any ideas?
>
> --
> Jim Canon
> Sr. Data Protection Analyst
> Global Technology Services - AHS
> Mylan
> 781 Chestnut Ridge Road
> Morgantown, WV 26505
>
> jim.ca...@mylan.com
>
> Phone: 800.826.9526 ext. 6236
> Direct: 304.554.6236
> Fax: 304.285.6404
> Mobile: 304.224.6815
>
>
>
> ==
> CONFIDENTIALITY NOTICE:  This e-mail message and all attachments
> transmitted with it may contain legally privileged, proprietary and/or
> confidential information intended solely for the use of the addressee.  If
> you are not the intended recipient, you are hereby notified that any review,
> dissemination, distribution, duplication or other use of this message and/or
> its attachments is strictly prohibited.  If you are not the intended
> recipient, please contact the sender by reply e-mail and destroy all copies
> of the original message and its attachments.  Thank you.
>
> ==
>
> ___
> Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>
>
___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu


Re: [Veritas-bu] NetBackup 7.1 media server backup issue

2011-05-06 Thread Jim . Canon
Hi Saran,

The backup selection is /

I have cross mount points selected. We don't follow NFS.

I can try splitting the backup into multiple streams, but I don't think
there is a file read issue. I can run (bpbkar -nocont / > /dev/null)
without any problems what so ever.

Thanks,

--
Jim Canon
Sr. Data Protection Analyst
Global Technology Services - AHS
Mylan
781 Chestnut Ridge Road
Morgantown, WV 26505

jim.ca...@mylan.com

Phone: 800.826.9526 ext. 6236
Direct: 304.554.6236
Fax: 304.285.6404
Mobile: 304.224.6815



From:   "Saran Brar" 
To: jim.ca...@mylan.com
Cc: veritas-bu@mailman.eng.auburn.edu
Date:   05/06/2011 01:20 AM
Subject:    Re: [Veritas-bu] NetBackup 7.1 media server backup issue



I have few questions:

- What is in the backup selection.
- Follow NFS and cross mount points selected
- Try splitting up the backup selection in multiple streams and pin point
exactly which file or folder is creating problem.

On Fri, May 6, 2011 at 9:15 AM,  wrote:

  Hi All,

  We're having a strange problem when the media server (AIX 6.1) is backing
  up itself. This problem only occurs when the media server attempts to
  back
  up itself. The media server can back up other clients just fine. Other
  media servers can back up this media server just fine.

  Here is the general output of the backup:

  05/05/2011 09:37:01 - begin writing
  05/05/2011 09:52:16 - Error bpbrm (pid=7405600) socket read failed: errno
  =
  119 - System call timed out
  05/05/2011 09:52:18 - Error bptm (pid=8912950) media manager terminated
  by
  parent process
  05/05/2011 09:52:27 - Info bpbkar (pid=9437276) done. status: 13: file
  read
  failed

  This isn't necessarily helpful, so I've enabled verbose logging level 5
  on
  both bpbkar and bpbrm. I'm not seeing any red flags.

  I have tried disabling shared memory, reducing the buffers to bare
  minimums, excluding what I thought were problem files, but it seems we
  are
  chasing a ghost. It is as if bpbkar stops sending data. The backup runs
  for
  a while, stops on a random file, and sits there until the job times out.

  Any ideas?

  --
  Jim Canon
  Sr. Data Protection Analyst
  Global Technology Services - AHS
  Mylan
  781 Chestnut Ridge Road
  Morgantown, WV 26505

  jim.ca...@mylan.com

  Phone: 800.826.9526 ext. 6236
  Direct: 304.554.6236
  Fax: 304.285.6404
  Mobile: 304.224.6815


  ==

  CONFIDENTIALITY NOTICE:  This e-mail message and all attachments
  transmitted with it may contain legally privileged, proprietary and/or
  confidential information intended solely for the use of the addressee.
  If you are not the intended recipient, you are hereby notified that any
  review, dissemination, distribution, duplication or other use of this
  message and/or its attachments is strictly prohibited.  If you are not
  the intended recipient, please contact the sender by reply e-mail and
  destroy all copies of the original message and its attachments.  Thank
  you.
  ==


  ___
  Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
  http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu




==
CONFIDENTIALITY NOTICE:  This e-mail message and all attachments transmitted 
with it may contain legally privileged, proprietary and/or confidential 
information intended solely for the use of the addressee.  If you are not the 
intended recipient, you are hereby notified that any review, dissemination, 
distribution, duplication or other use of this message and/or its attachments 
is strictly prohibited.  If you are not the intended recipient, please contact 
the sender by reply e-mail and destroy all copies of the original message and 
its attachments.  Thank you.
==

___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu


Re: [Veritas-bu] NetBackup 7.1 media server backup issue

2011-05-06 Thread Jim . Canon
Girish,

A file read error is what it appears, however (bpbkar -nocont /
> /dev/null) completes with status 0.

Here is where it gets strange. I've also tried creating a storage unit that
uses /dev/null as it's basic disk pool. I've then set the backup policy to
use this storage unit. This backup fails with the same error as if I use a
storage unit and disk pool pointing to our data domain.

Thanks,

--
Jim Canon
Sr. Data Protection Analyst
Global Technology Services - AHS
Mylan
781 Chestnut Ridge Road
Morgantown, WV 26505

jim.ca...@mylan.com

Phone: 800.826.9526 ext. 6236
Direct: 304.554.6236
Fax: 304.285.6404
Mobile: 304.224.6815



From:   "Girish Jorapurkar" 
To: veritas-bu@mailman.eng.auburn.edu, jim.ca...@mylan.com
Date:   05/05/2011 10:01 PM
Subject:    Re: [Veritas-bu] NetBackup 7.1 media server backup issue
Sent by:veritas-bu-boun...@mailman.eng.auburn.edu



If I am understanding it correctly, an error occurred while reading a file.

> 05/05/2011 09:52:27 - Info bpbkar (pid=9437276) done.
> status: 13: file read failed

That is strange. See if there is any storage (volume manager or disk I/O)
related error/fault reported at the time of bpbkar failure.

Also, you can try bpbkar null test - just to ensure that data can be read
without any random read failures:

bpbkar -nocont  > /dev/null

Hope that helps,
/Girish

--- On Fri, 5/6/11, jim.ca...@mylan.com  wrote:

> From: jim.ca...@mylan.com 
> Subject: [Veritas-bu] NetBackup 7.1 media server backup issue
> To: veritas-bu@mailman.eng.auburn.edu
> Date: Friday, May 6, 2011, 6:45 AM
>
> Hi All,
>
> We're having a strange problem when the media server (AIX
> 6.1) is backing
> up itself. This problem only occurs when the media server
> attempts to back
> up itself. The media server can back up other clients just
> fine. Other
> media servers can back up this media server just fine.
>
> Here is the general output of the backup:
>
> 05/05/2011 09:37:01 - begin writing
> 05/05/2011 09:52:16 - Error bpbrm (pid=7405600) socket read
> failed: errno =
> 119 - System call timed out
> 05/05/2011 09:52:18 - Error bptm (pid=8912950) media
> manager terminated by
> parent process
> 05/05/2011 09:52:27 - Info bpbkar (pid=9437276) done.
> status: 13: file read
> failed
>
> This isn't necessarily helpful, so I've enabled verbose
> logging level 5 on
> both bpbkar and bpbrm. I'm not seeing any red flags.
>
> I have tried disabling shared memory, reducing the buffers
> to bare
> minimums, excluding what I thought were problem files, but
> it seems we are
> chasing a ghost. It is as if bpbkar stops sending data. The
> backup runs for
> a while, stops on a random file, and sits there until the
> job times out.
>
> Any ideas?
>
> --
> Jim Canon
> Sr. Data Protection Analyst
> Global Technology Services - AHS
> Mylan
> 781 Chestnut Ridge Road
> Morgantown, WV 26505
>
> jim.ca...@mylan.com
>
> Phone: 800.826.9526 ext. 6236
> Direct: 304.554.6236
> Fax: 304.285.6404
> Mobile: 304.224.6815
>
>
>
==

> CONFIDENTIALITY NOTICE:  This e-mail message and all
> attachments transmitted with it may contain legally
> privileged, proprietary and/or confidential information
> intended solely for the use of the addressee.  If you
> are not the intended recipient, you are hereby notified that
> any review, dissemination, distribution, duplication or
> other use of this message and/or its attachments is strictly
> prohibited.  If you are not the intended recipient,
> please contact the sender by reply e-mail and destroy all
> copies of the original message and its attachments.
> Thank you.
>
==

>
> ___
> Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>
___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu




==
CONFIDENTIALITY NOTICE:  This e-mail message and all attachments transmitted 
with it may contain legally privileged, proprietary and/or confidential 
information intended solely for the use of the addressee.  If you are not the 
intended recipient, you are hereby notified that any review, dissemination, 
distribution, duplication or other use of this message and/or its attachments 
is strictly prohibited.  If you are not the intended recipient, please contact 
the sender by reply e-mail and destroy all copies of the original message and 
its attachments.  Thank you.
==

___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu


Re: [Veritas-bu] NetBackup 7.1 media server backup issue

2011-05-06 Thread Rusty.Major
When we've encountered these issues and the logs were no help, we had to 
narrow down the problem by creating multiple backups or streams. This 
approach allowed us to narrow down the culprit by eventually closing in on 
a bad file during the backup.

Rusty Major, MCSE, BCFP, VCS ▪ Sr. Storage Engineer ▪ SunGard 
Availability Services ▪ 757 N. Eldridge Suite 200, Houston TX 77079 ▪ 
281-584-4693
Keeping People and Information Connected® ▪ 
http://availability.sungard.com/ 
P Think before you print 
CONFIDENTIALITY:  This e-mail (including any attachments) may contain 
confidential, proprietary and privileged information, and unauthorized 
disclosure or use is prohibited.  If you received this e-mail in error, 
please notify the sender and delete this e-mail from your system. 



 
Sent by: 
05/06/2011 10:41 AM

To
Girish Jorapurkar 
cc

Subject
Re: [Veritas-bu] NetBackup 7.1 media server backup issue






Girish,

A file read error is what it appears, however (bpbkar -nocont /
> /dev/null) completes with status 0.

Here is where it gets strange. I've also tried creating a storage unit 
that
uses /dev/null as it's basic disk pool. I've then set the backup policy to
use this storage unit. This backup fails with the same error as if I use a
storage unit and disk pool pointing to our data domain.

Thanks,

--
Jim Canon
Sr. Data Protection Analyst
Global Technology Services - AHS
Mylan
781 Chestnut Ridge Road
Morgantown, WV 26505

jim.ca...@mylan.com

Phone: 800.826.9526 ext. 6236
Direct: 304.554.6236
Fax: 304.285.6404
Mobile: 304.224.6815



From:"Girish Jorapurkar" 
To:  veritas-bu@mailman.eng.auburn.edu, jim.ca...@mylan.com
Date:05/05/2011 10:01 PM
Subject:         Re: [Veritas-bu] NetBackup 7.1 media server 
backup issue
Sent by: veritas-bu-boun...@mailman.eng.auburn.edu



If I am understanding it correctly, an error occurred while reading a 
file.

> 05/05/2011 09:52:27 - Info bpbkar (pid=9437276) done.
> status: 13: file read failed

That is strange. See if there is any storage (volume manager or disk I/O)
related error/fault reported at the time of bpbkar failure.

Also, you can try bpbkar null test - just to ensure that data can be read
without any random read failures:

bpbkar -nocont  > /dev/null

Hope that helps,
/Girish

--- On Fri, 5/6/11, jim.ca...@mylan.com  wrote:

> From: jim.ca...@mylan.com 
> Subject: [Veritas-bu] NetBackup 7.1 media server backup issue
> To: veritas-bu@mailman.eng.auburn.edu
> Date: Friday, May 6, 2011, 6:45 AM
>
> Hi All,
>
> We're having a strange problem when the media server (AIX
> 6.1) is backing
> up itself. This problem only occurs when the media server
> attempts to back
> up itself. The media server can back up other clients just
> fine. Other
> media servers can back up this media server just fine.
>
> Here is the general output of the backup:
>
> 05/05/2011 09:37:01 - begin writing
> 05/05/2011 09:52:16 - Error bpbrm (pid=7405600) socket read
> failed: errno =
> 119 - System call timed out
> 05/05/2011 09:52:18 - Error bptm (pid=8912950) media
> manager terminated by
> parent process
> 05/05/2011 09:52:27 - Info bpbkar (pid=9437276) done.
> status: 13: file read
> failed
>
> This isn't necessarily helpful, so I've enabled verbose
> logging level 5 on
> both bpbkar and bpbrm. I'm not seeing any red flags.
>
> I have tried disabling shared memory, reducing the buffers
> to bare
> minimums, excluding what I thought were problem files, but
> it seems we are
> chasing a ghost. It is as if bpbkar stops sending data. The
> backup runs for
> a while, stops on a random file, and sits there until the
> job times out.
>
> Any ideas?
>
> --
> Jim Canon
> Sr. Data Protection Analyst
> Global Technology Services - AHS
> Mylan
> 781 Chestnut Ridge Road
> Morgantown, WV 26505
>
> jim.ca...@mylan.com
>
> Phone: 800.826.9526 ext. 6236
> Direct: 304.554.6236
> Fax: 304.285.6404
> Mobile: 304.224.6815
>
>
>
==

> CONFIDENTIALITY NOTICE:  This e-mail message and all
> attachments transmitted with it may contain legally
> privileged, proprietary and/or confidential information
> intended solely for the use of the addressee.  If you
> are not the intended recipient, you are hereby notified that
> any review, dissemination, distribution, duplication or
> other use of this message and/or its attachments is strictly
> prohibited.  If you are not the intended recipient,
> please contact the sender by reply e-mail and destroy all
> copies of the original message and its attachments.
> Thank you.
>
==

>
&

Re: [Veritas-bu] NetBackup 7.1 media server backup issue

2011-05-06 Thread Len Boyle
Jim, 

You might want to look at locks. Are there any message that help in the aix 
system logs.
Trying checking your client settings for waiting for locked or in use files for 
the aix media server. 
I do not know if this is the problem, just a thought.

len


-Original Message-
From: veritas-bu-boun...@mailman.eng.auburn.edu 
[mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of 
jim.ca...@mylan.com
Sent: Thursday, May 05, 2011 9:16 PM
To: veritas-bu@mailman.eng.auburn.edu
Subject: [Veritas-bu] NetBackup 7.1 media server backup issue


Hi All,

We're having a strange problem when the media server (AIX 6.1) is backing up 
itself. This problem only occurs when the media server attempts to back up 
itself. The media server can back up other clients just fine. Other media 
servers can back up this media server just fine.

Here is the general output of the backup:

05/05/2011 09:37:01 - begin writing
05/05/2011 09:52:16 - Error bpbrm (pid=7405600) socket read failed: errno =
119 - System call timed out
05/05/2011 09:52:18 - Error bptm (pid=8912950) media manager terminated by 
parent process
05/05/2011 09:52:27 - Info bpbkar (pid=9437276) done. status: 13: file read 
failed

This isn't necessarily helpful, so I've enabled verbose logging level 5 on both 
bpbkar and bpbrm. I'm not seeing any red flags.

I have tried disabling shared memory, reducing the buffers to bare minimums, 
excluding what I thought were problem files, but it seems we are chasing a 
ghost. It is as if bpbkar stops sending data. The backup runs for a while, 
stops on a random file, and sits there until the job times out.

Any ideas?

--
Jim Canon
Sr. Data Protection Analyst
Global Technology Services - AHS
Mylan
781 Chestnut Ridge Road
Morgantown, WV 26505

jim.ca...@mylan.com

Phone: 800.826.9526 ext. 6236
Direct: 304.554.6236
Fax: 304.285.6404
Mobile: 304.224.6815


==
CONFIDENTIALITY NOTICE:  This e-mail message and all attachments transmitted 
with it may contain legally privileged, proprietary and/or confidential 
information intended solely for the use of the addressee.  If you are not the 
intended recipient, you are hereby notified that any review, dissemination, 
distribution, duplication or other use of this message and/or its attachments 
is strictly prohibited.  If you are not the intended recipient, please contact 
the sender by reply e-mail and destroy all copies of the original message and 
its attachments.  Thank you.
==

___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu 
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu


___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu


Re: [Veritas-bu] NetBackup 7.1 media server backup issue

2011-05-06 Thread Jim . Canon
Hi Len,

This too was something we considered. The "locked file action" in the media
server client settings is set to "skip".

We enabled debug level system logging and got nothing of use from aix.

On a side note - I got 1 random successful full backup today. I immediately
ran another full, and got the same old error. I have also tried writing
directly to tape - same error.

--
Jim Canon
Sr. Data Protection Analyst
Global Technology Services - AHS
Mylan
781 Chestnut Ridge Road
Morgantown, WV 26505

jim.ca...@mylan.com

Phone: 800.826.9526 ext. 6236
Direct: 304.554.6236
Fax: 304.285.6404
Mobile: 304.224.6815



From:   "Len Boyle" 
To: "jim.ca...@mylan.com" ,
"veritas-bu@mailman.eng.auburn.edu"

Date:   05/06/2011 04:31 PM
Subject:    Re: [Veritas-bu] NetBackup 7.1 media server backup issue
Sent by:veritas-bu-boun...@mailman.eng.auburn.edu



Jim,

You might want to look at locks. Are there any message that help in the aix
system logs.
Trying checking your client settings for waiting for locked or in use files
for the aix media server.
I do not know if this is the problem, just a thought.

len


-Original Message-
From: veritas-bu-boun...@mailman.eng.auburn.edu [
mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of
jim.ca...@mylan.com
Sent: Thursday, May 05, 2011 9:16 PM
To: veritas-bu@mailman.eng.auburn.edu
Subject: [Veritas-bu] NetBackup 7.1 media server backup issue


Hi All,

We're having a strange problem when the media server (AIX 6.1) is backing
up itself. This problem only occurs when the media server attempts to back
up itself. The media server can back up other clients just fine. Other
media servers can back up this media server just fine.

Here is the general output of the backup:

05/05/2011 09:37:01 - begin writing
05/05/2011 09:52:16 - Error bpbrm (pid=7405600) socket read failed: errno =
119 - System call timed out
05/05/2011 09:52:18 - Error bptm (pid=8912950) media manager terminated by
parent process
05/05/2011 09:52:27 - Info bpbkar (pid=9437276) done. status: 13: file read
failed

This isn't necessarily helpful, so I've enabled verbose logging level 5 on
both bpbkar and bpbrm. I'm not seeing any red flags.

I have tried disabling shared memory, reducing the buffers to bare
minimums, excluding what I thought were problem files, but it seems we are
chasing a ghost. It is as if bpbkar stops sending data. The backup runs for
a while, stops on a random file, and sits there until the job times out.

Any ideas?

--
Jim Canon
Sr. Data Protection Analyst
Global Technology Services - AHS
Mylan
781 Chestnut Ridge Road
Morgantown, WV 26505

jim.ca...@mylan.com

Phone: 800.826.9526 ext. 6236
Direct: 304.554.6236
Fax: 304.285.6404
Mobile: 304.224.6815


==

CONFIDENTIALITY NOTICE:  This e-mail message and all attachments
transmitted with it may contain legally privileged, proprietary and/or
confidential information intended solely for the use of the addressee.  If
you are not the intended recipient, you are hereby notified that any
review, dissemination, distribution, duplication or other use of this
message and/or its attachments is strictly prohibited.  If you are not the
intended recipient, please contact the sender by reply e-mail and destroy
all copies of the original message and its attachments.  Thank you.
==


___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu


___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu




==
CONFIDENTIALITY NOTICE:  This e-mail message and all attachments transmitted 
with it may contain legally privileged, proprietary and/or confidential 
information intended solely for the use of the addressee.  If you are not the 
intended recipient, you are hereby notified that any review, dissemination, 
distribution, duplication or other use of this message and/or its attachments 
is strictly prohibited.  If you are not the intended recipient, please contact 
the sender by reply e-mail and destroy all copies of the original message and 
its attachments.  Thank you.
==

___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu


Re: [Veritas-bu] NetBackup 7.1 media server backup issue

2011-05-06 Thread Jim . Canon
Hi Rusty,

I have been trying a similar approach using the exclude lists. I haven't
been able to gather enough data to say for certain that a specific group of
files is causing the problem. There doesn't seem to be a specific file
causing the issue.

--
Jim Canon
Sr. Data Protection Analyst
Global Technology Services - AHS
Mylan
781 Chestnut Ridge Road
Morgantown, WV 26505

jim.ca...@mylan.com

Phone: 800.826.9526 ext. 6236
Direct: 304.554.6236
Fax: 304.285.6404
Mobile: 304.224.6815



From:   rusty.ma...@sungard.com
To: jim.ca...@mylan.com
Cc: veritas-bu@mailman.eng.auburn.edu,
veritas-bu-boun...@mailman.eng.auburn.edu
Date:   05/06/2011 04:04 PM
Subject:    Re: [Veritas-bu] NetBackup 7.1 media server backup issue
Sent by:veritas-bu-boun...@mailman.eng.auburn.edu



When we've encountered these issues and the logs were no help, we had to
narrow down the problem by creating multiple backups or streams. This
approach allowed us to narrow down the culprit by eventually closing in on
a bad file during the backup.

Rusty Major, MCSE, BCFP, VCS ▪ Sr. Storage Engineer ▪ SunGard Availability
Services ▪ 757 N. Eldridge Suite 200, Houston TX 77079 ▪ 281-584-4693
Keeping People and Information Connected® ▪
http://availability.sungard.com/
P Think before you print
CONFIDENTIALITY:  This e-mail (including any attachments) may contain
confidential, proprietary and privileged information, and unauthorized
disclosure or use is prohibited.  If you received this e-mail in error,
please notify the sender and delete this e-mail from your system.

   
  
 Sent by:  
  To 
 Girish Jorapurkar 
   
 05/06/2011 10:41 AMcc 
 
   Subject 
         Re: [Veritas-bu] NetBackup
         7.1 media server backup issue 
   
   
   
   
   
   
   





Girish,

A file read error is what it appears, however (bpbkar -nocont /
> /dev/null) completes with status 0.

Here is where it gets strange. I've also tried creating a storage unit that
uses /dev/null as it's basic disk pool. I've then set the backup policy to
use this storage unit. This backup fails with the same error as if I use a
storage unit and disk pool pointing to our data domain.

Thanks,

--
Jim Canon
Sr. Data Protection Analyst
Global Technology Services - AHS
Mylan
781 Chestnut Ridge Road
Morgantown, WV 26505

jim.ca...@mylan.com

Phone: 800.826.9526 ext. 6236
Direct: 304.554.6236
Fax: 304.285.6404
Mobile: 304.224.6815



From: "Girish Jorapurkar" 
To: veritas-bu@mailman.eng.auburn.edu, jim.ca...@mylan.com
Date: 05/05/2011 10:01 PM
Subject:         Re: [Veritas-bu] NetBackup 7.1 media server backup
issue
Sent by: veritas-bu-boun...@mailman.eng.auburn.edu



If I am understanding it correctly, an error occurred while reading a file.

> 05/05/2011 09:52:27 - Info bpbkar (pid=9437276) done.
> status: 13: file read failed

That is strange. See if there is any storage (volume manager or disk I/O)
related error/fault reported at the time of bpbkar failure.

Also, you can try bpbkar null test - just to ensure that data can be read
without any random read failures:

bpbkar -nocont  > /dev/null

Hope that helps,
/Girish

--- On Fri, 5/6/11, jim.ca...@mylan.com  wrote:

> From: jim.ca...@mylan.com 
> Subject: [Veritas-bu] NetBackup 7.1 media server backup issue
> To: veritas-bu@mailman.eng.auburn.edu
> Date: Friday, May 6, 2011, 6:45 AM
>
> Hi All,
>
> We're having a strange problem when the media server (AIX
> 6.1) is backing
> up itself. This problem only occurs when the media server
> attempts to back
> up itself. The media server can back up other clients just
> fine. Other
> media servers can back up this media server just fine.
>
> Here is the general output of the back