Re: working with storage definitions

2017-09-01 Thread Chris Hoogendyk 

OK.

So, how does the planner plan for what goes on each storage? That is, if I set up a pseudo tapetype 
for global and give it a length of, say, 12.5TB; how will it know that the uncompressed DLEs are 
targeted to the LTO7 and the compressed DLEs are targeted at the LTO6? What if the proportions came 
out wrong (say, 8TB of DLEs intended for the LTO7 and 4.5TB of DLEs intened for the LTO6), but the 
total was within the 12.5TB?


Also, there is no commentary in the Amanda email report indicating where each DLE went, just the 
overall amount of data that was written to each tape. I presume DLEs are going to the intended 
tapes, but . . . .



On 9/1/17 1:54 PM, Jean-Louis Martineau wrote:

It is an unimplemented features.
The planner do not look at the storage, it still use the global tapetype and 
runtapes.
Add a new tapetype with a length that is the sum of the two storage tapetype length * runtapes (or 
the maximum dump size you want in one run)

Set the global tapetype to that new tapetype and set the global runtapes to 1.

Jean-Louis

On 01/09/17 01:39 PM, Chris Hoogendyk wrote:

oops. Sorry about that. Attached amdump.20170831233001


On 9/1/17 1:25 PM, Jean-Louis Martineau wrote:
> I want the amdump. file, it is in the same directory as 
log.20170831233001.0
>
> On 01/09/17 01:22 PM, Chris Hoogendyk wrote:
>> attached trace log identified in the following debug file.
>>
>> amanda@marlin:~/daily$ less 
/tmp/amanda/server/daily/amdump.20170831233001.debug
>>
>> Thu Aug 31 23:30:01.546380821 2017: pid 1898: thd-0x1013000: amdump: pid 
1898 ruid 555 euid 555
>> version 3.4.5: start at Thu Aug 31 23:30:01 2017
>> Thu Aug 31 23:30:01.546931671 2017: pid 1898: thd-0x1013000: amdump: 
Arguments: daily
>> Thu Aug 31 23:30:01.547756321 2017: pid 1898: thd-0x1013000: amdump: reading 
config file
>> /usr/local/etc/amanda/daily/amanda.conf
>> Thu Aug 31 23:30:01.680034064 2017: pid 1898: thd-0x1013000: amdump: pid 
1898 ruid 555 euid 555
>> version 3.4.5: rename at Thu Aug 31 23:30:01 2017
>> Thu Aug 31 23:30:01.680717936 2017: pid 1898: thd-0x1013000: amdump: 
*beginning trace log:
>> /usr/local/etc/amanda/daily/log/log.20170831233001.0*
>> Thu Aug 31 23:30:01.680791773 2017: pid 1898: thd-0x1013000: amdump: 
beginning amdump log
>> Thu Aug 31 23:30:01.681288047 2017: pid 1898: thd-0x1013000: amdump:
>> /usr/local/share/perl/5.18.2/Amanda/Amdump.pm:97:info:203 The timestamp 
is '20170831233001'
>> Thu Aug 31 23:30:01.681713007 2017: pid 1898: thd-0x1013000: amdump:
>> /usr/local/share/perl/5.18.2/Amanda/Amdump.pm:103:info:201 The amdump 
trace file is
>> '/usr/local/etc/amanda/daily/log/amdump.20170831233001'
>> Thu Aug 31 23:30:01.682060951 2017: pid 1898: thd-0x1013000: amdump:
>> /usr/local/share/perl/5.18.2/Amanda/Amdump.pm:109:info:200 The log file 
is
>> '/usr/local/etc/amanda/daily/log/log.20170831233001.0'
>> Thu Aug 31 23:30:01.682475809 2017: pid 1898: thd-0x1013000: amdump: 
invoking planner | driver
>> Thu Aug 31 23:30:01.684324606 2017: pid 1898: thd-0x1013000: amdump:  
planner: 1899
>> Thu Aug 31 23:30:01.684758762 2017: pid 1899: thd-0x1013000: amdump: exec:
>> /usr/local/libexec/amanda/planner daily --starttime 20170831233001 
--log-filename
>> /usr/local/etc/amanda/daily/log/log.20170831233001.0
>> Thu Aug 31 23:30:01.684983591 2017: pid 1899: thd-0x1013000: amdump: exec
>> /usr/local/libexec/amanda/planner daily --starttime 20170831233001 
--log-filename
>> /usr/local/etc/amanda/daily/log/log.20170831233001.0
>> Thu Aug 31 23:30:01.685602431 2017: pid 1898: thd-0x1013000: amdump:  
driver: 1900
>> Thu Aug 31 23:30:01.685953697 2017: pid 1900: thd-0x1013000: amdump: exec:
>> /usr/local/libexec/amanda/driver daily --log-filename
>> /usr/local/etc/amanda/daily/log/log.20170831233001.0
>> Thu Aug 31 23:30:01.686152981 2017: pid 1900: thd-0x1013000: amdump: exec
>> /usr/local/libexec/amanda/driver daily --log-filename
>> /usr/local/etc/amanda/daily/log/log.20170831233001.0
>> Thu Aug 31 23:57:34.602971243 2017: pid 1898: thd-0x1013000: amdump: planner 
finished with exit
>> code 0
>> Fri Sep 01 05:47:20.955904196 2017: pid 1898: thd-0x1013000: amdump: driver 
finished with exit
>> code 0
>> Fri Sep 01 05:47:20.956138752 2017: pid 1898: thd-0x1013000: amdump: running 
amreport
>> Fri Sep 01 05:47:20.956268742 2017: pid 1898: thd-0x1013000: amdump: Running
>> /usr/local/sbin/amreport daily --from-amdump -l 
/usr/local/etc/amanda/daily/log/log.20170831233001.0

>> Fri Sep 01 05:47:22.808415301 2017: pid 1898: thd-0x1013000: amdump: 
/usr/local/sbin/amreport
>> exited with code 14
>> Fri Sep 01 05:47:22.808511674 2017: pid 1898: thd-0x1013000: amdump: 
trimming old trace logs
>> Fri Sep 01 05:47:22.808573795 2017: pid 1898: thd-0x1013000: amdump: Running
>> /usr/local/libexec/amanda/amtrmlog daily
>> Fri Sep 01 05:47:22.926108466 2017: pid 1898: thd-0x1013000: amdump:
>> /usr/local/libexec/amanda/amtrmlog exited with code 0
>> Fri Sep 01 05:47:22.926212127 2017: pid 1898: thd-0x1013000:

Re: working with storage definitions

2017-09-01 Thread Jean-Louis Martineau 

On 01/09/17 02:10 PM, Chris Hoogendyk wrote:

OK.

So, how does the planner plan for what goes on each storage?

It doesn't, it use the size and plan to dump less than that.


That is, if I set up a pseudo tapetype
for global and give it a length of, say, 12.5TB; how will it know that 
the uncompressed DLEs are
targeted to the LTO7 and the compressed DLEs are targeted at the LTO6? 
What if the proportions came
out wrong (say, 8TB of DLEs intended for the LTO7 and 4.5TB of DLEs 
intened for the LTO6), but the

total was within the 12.5TB?

That's the problem, some DLEs will fail unless you have enough holdingdisk.
I never liked the idea that amanda delay some full dump because it do 
not have enough tape space, It get worse at every days
The solution to never hit that problem is to always provide enough tape, 
you could increase the runtapes of both storage.




Also, there is no commentary in the Amanda email report indicating 
where each DLE went, just the
overall amount of data that was written to each tape. I presume DLEs 
are going to the intended

tapes, but . . . .

Where would you like the information to be printed.

You can run 'amadmin CONF find', it should list the storage for each DLEs.

Jean-Louis
This message is the property of CARBONITE, INC. and may contain confidential or 
privileged information.
If this message has been delivered to you by mistake, then do not copy or 
deliver this message to anyone.  Instead, destroy it and notify me by reply 
e-mail

Re: working with storage definitions

2017-09-01 Thread Chris Hoogendyk 
hmm. That is kind of a problem. I should have more than sufficient space now, but the allocation of 
where it goes could overflow if the planning is blind to the storage allocations. Maybe that is 
something to be developed?


As to where to report the storage use, the Amanda report doesn't have much space for another column. 
However, the Dump Summary could be broken into separate Dump Summary tables for each storage. I can 
obviously look at my configuration files to see where things go, but it would be reassuring to have 
the Amanda report reflect that.


As for my current configuration, I created a pseudo tapetype that has a length equal to the sum of 
what I expect to be able to write to one LTO6 plus one LTO7 tape. I put that as the global tapetype, 
with runtapes=1. Each storage still references it's own tpchanger and tapetype.



On 9/1/17 2:31 PM, Jean-Louis Martineau wrote:

On 01/09/17 02:10 PM, Chris Hoogendyk wrote:
> OK.
>
> So, how does the planner plan for what goes on each storage?
It doesn't, it use the size and plan to dump less than that.

> That is, if I set up a pseudo tapetype
> for global and give it a length of, say, 12.5TB; how will it know that
> the uncompressed DLEs are
> targeted to the LTO7 and the compressed DLEs are targeted at the LTO6?
> What if the proportions came
> out wrong (say, 8TB of DLEs intended for the LTO7 and 4.5TB of DLEs
> intened for the LTO6), but the
> total was within the 12.5TB?
That's the problem, some DLEs will fail unless you have enough holdingdisk.
I never liked the idea that amanda delay some full dump because it do
not have enough tape space, It get worse at every days
The solution to never hit that problem is to always provide enough tape,
you could increase the runtapes of both storage.

>
> Also, there is no commentary in the Amanda email report indicating
> where each DLE went, just the
> overall amount of data that was written to each tape. I presume DLEs
> are going to the intended
> tapes, but . . . .
Where would you like the information to be printed.

You can run 'amadmin CONF find', it should list the storage for each DLEs.

Jean-Louis


*Disclaimer*

This message is the property of *CARBONITE, INC.*  and may contain 
confidential or privileged information.


If this message has been delivered to you by mistake, then do not copy or deliver this message to 
anyone. Instead, destroy it and notify me by reply e-mail.




--
---

Chris Hoogendyk

-
   O__   Systems Administrator
  c/ /'_ --- Biology & Geosciences Departments
 (*) \(*) -- 315 Morrill Science Center
~~ - University of Massachusetts, Amherst



---

Erdös 4

Re: working with storage definitions

2017-09-01 Thread Jean-Louis Martineau 

On 01/09/17 03:16 PM, Chris Hoogendyk wrote:
hmm. That is kind of a problem. I should have more than sufficient 
space now, but the allocation of
where it goes could overflow if the planning is blind to the storage 
allocations. Maybe that is

something to be developed?

As I already wrote, It is an unimplemented features.


As to where to report the storage use, the Amanda report doesn't have 
much space for another column.
However, the Dump Summary could be broken into separate Dump Summary 
tables for each storage. I can
obviously look at my configuration files to see where things go, but 
it would be reassuring to have

the Amanda report reflect that.
The Dump Summary is for what is dumped,they could still be in holding 
disk and not on tape.


Maybe we could completetly rewrite the columnspec to be more like a printf
  "%12.0H %-11.0D %2L  %S

H is for host
D is for disk
L is for level
...
S is for storage

We can then add many fields.



As for my current configuration, I created a pseudo tapetype that has 
a length equal to the sum of
what I expect to be able to write to one LTO6 plus one LTO7 tape. I 
put that as the global tapetype,
with runtapes=1. Each storage still references it's own tpchanger and 
tapetype.



On 9/1/17 2:31 PM, Jean-Louis Martineau wrote:
> On 01/09/17 02:10 PM, Chris Hoogendyk wrote:
> > OK.
> >
> > So, how does the planner plan for what goes on each storage?
> It doesn't, it use the size and plan to dump less than that.
>
> > That is, if I set up a pseudo tapetype
> > for global and give it a length of, say, 12.5TB; how will it know that
> > the uncompressed DLEs are
> > targeted to the LTO7 and the compressed DLEs are targeted at the LTO6?
> > What if the proportions came
> > out wrong (say, 8TB of DLEs intended for the LTO7 and 4.5TB of DLEs
> > intened for the LTO6), but the
> > total was within the 12.5TB?
> That's the problem, some DLEs will fail unless you have enough 
holdingdisk.

> I never liked the idea that amanda delay some full dump because it do
> not have enough tape space, It get worse at every days
> The solution to never hit that problem is to always provide enough tape,
> you could increase the runtapes of both storage.
>
> >
> > Also, there is no commentary in the Amanda email report indicating
> > where each DLE went, just the
> > overall amount of data that was written to each tape. I presume DLEs
> > are going to the intended
> > tapes, but . . . .
> Where would you like the information to be printed.
>
> You can run 'amadmin CONF find', it should list the storage for each 
DLEs.

>
> Jean-Louis
>
>
> *Disclaimer*
>
> This message is the property of *CARBONITE, INC.* 
> 
and may contain

> confidential or privileged information.
>
> If this message has been delivered to you by mistake, then do not 
copy or deliver this message to

> anyone. Instead, destroy it and notify me by reply e-mail.
>

--
---

Chris Hoogendyk

-
O__  Systems Administrator
c/ /'_ --- Biology & Geosciences Departments
(*) \(*) -- 315 Morrill Science Center
~~ - University of Massachusetts, Amherst



---

Erdös 4

This message is the property of CARBONITE, INC. and may contain confidential or 
privileged information.
If this message has been delivered to you by mistake, then do not copy or 
deliver this message to anyone.  Instead, destroy it and notify me by reply 
e-mail