Re: [robinhood-support] Inheritance of action parameters

Thu, 16 Feb 2017 23:51:23 -0800 

On 17/02/17 17:14, Gerald Hofer wrote:


Hi,


I am testing a configuration that should sort files into different 
archive ids based on location and I want to use a different parameters 
for the copytool based on file size. But I think I ran into a bug in 
that configuration.



My understanding is that the parameters should be done based on the 
following order:


1) As default parameters for a policy
2) In policy triggers
3) In policy rules
4) In fileclass definitions


So these are the different levels in my example:

1) The parameters sections has a default and defines the command:

lhsm_archive_parameters {
    nb_threads = 8;
## archive 1000 files max at once
#    max_action_count = 1000;
#    max_action_volume = 1TB;

    # suspend policy run if action error rate > 50% (after 100 errors)
    suspend_error_pct = 50%;
    suspend_error_min= 100;

    # overrides policy default action

    action = cmd("lfs hsm_archive --archive {archive_id} --data {data} 
/gdatat1/.lustre/fid/{fid}");


    # default action parameters
    action_params {
        archive_id = 1;
        data = t2;
    }
}

2) No action params in the trigger:

lhsm_archive_trigger {
    trigger_on = periodic;
    check_interval = 1min;
}



3) The rules section has currently no parameters either ( I had the 
archive ids on that level earlier):


lhsm_archive_rules {
    ignore_fileclass = empty_files;
    ignore_fileclass = no_archive;



    rule archive_siteA {
        target_fileclass = siteA;

        condition { last_mod >= 1min }
#        action_params { archive_id = 1;
#                       data = "t1";
#       }
    }

    rule archive_siteB {
        target_fileclass = siteB;

        condition { last_mod >= 1min }
#        action_params { archive_id = 2; }
    }

    rule archive_siteAB {
        target_fileclass = siteAB;

        condition { last_mod >= 1min }
#        action_params { archive_id = 3; }
    }


    # fallback rule
    rule default {
        action_params { archive_id = 1; }
        condition { last_mod >= 1min }

    }
}


4) The file class definitions have now the different parameters:


FileClass empty_files {
    definition { type == file and size == 0 }
    lhsm_archive_action_params { data = "t1"; }
    # report = yes (default)
}
FileClass small_files {
    definition { type == file and size > 0 and size <= 16MB }
    lhsm_archive_action_params { data = "t1"; }
    # report = yes (default)
}
FileClass std_files {
    definition { type == file and size > 16MB and size <= 1GB }
    lhsm_archive_action_params { data = "t2"; }
}
FileClass big_files {
    definition { type == file and size > 1GB }
    lhsm_archive_action_params { data = "t2"; }
}
FileClass largedir {
    definition { type == directory and dircount > 10000 }
}
FileClass siteA {
    definition { tree == "/gdatat1/siteA" }
    lhsm_archive_action_params { archive_id = 1; }
}
FileClass siteB {
    definition { tree == "/gdatat1/siteB" }
    lhsm_archive_action_params { archive_id = 2;
                                 data = "t1"; }
}
FileClass siteAB {
    definition { tree == "/gdatat1/siteAB" }
    lhsm_archive_action_params { archive_id = 3; }
}
FileClass no_archive {
    definition { tree == "/gdatat1/no_archive" }
}



The observation is that FileClass definitions small_files, std_files 
and big_files does not seem to set the {data} parameter.
This particular configurations does test that the {data} parameter in 
principal gets set in a FileClass. If I write a file into 
/gdatat1/siteB the file gets migrated to archive_id 2 and data "t1". 
So both parameters are overwritten from the default archive_id 1 and 
"t2".
But if I am writing a small file into /gdatat1/siteAB the {data} 
parameter does not get overwritten from the default by the small_files 
FileClass.



This is the full debug log with a small file:



2017/02/17 18:18:58 [2499/4] lhsm_archive | Checking if entry 
/gdatat1/siteAB/test/resolv.conf matches policy rules
2017/02/17 18:18:58 [2499/11] lhsm_archive | End of list (1 entries 
returned)
2017/02/17 18:18:58 [2499/11] lhsm_archive | Waiting for the end of 
current pass: still 1 entries pending (0 in queue, 1 being processed). 
Last action 0s ago.
2017/02/17 18:18:58 [2499/4] lhsm_archive | Considering entry 
[0x200001b70:0x14d:0x0]

2017/02/17 18:18:58 [2499/4] Policy | Matching 'no_archive': rc=0

2017/02/17 18:18:58 [2499/4] Policy | Entry [0x200001b70:0x14d:0x0] 
matches target file class 'siteAB' of policy 'archive_siteAB'
2017/02/17 18:18:58 [2499/4] lhsm_archive | Entry 
/gdatat1/siteAB/test/resolv.conf matches the condition for policy rule 
'archive_siteAB'.

2017/02/17 18:18:58 [2499/4] params | 't2'->'t2' in parameter data='t2'
2017/02/17 18:18:58 [2499/4] params | '3'->'3' in parameter archive_id='3'

2017/02/17 18:18:58 [2499/4] lhsm_archive | Executing policy action on: 
0x200001b70:0x14d:0x0 (/gdatat1/siteAB/test/resolv.conf)
2017/02/17 18:18:58 [2499/4] lhsm_archive | [0x200001b70:0x14d:0x0]: 
action_params: data=t2, archive_id=3

2017/02/17 18:18:58 [2499/4] params | 'lfs'->'lfs' in action command 'lfs'

2017/02/17 18:18:58 [2499/4] params | [0] 'lfs'->'lfs' in action command 
'lfs'
2017/02/17 18:18:58 [2499/4] params | 'hsm_archive'->'hsm_archive' in 
action command 'lfs'
2017/02/17 18:18:58 [2499/4] params | [1] 'hsm_archive'->'hsm_archive' 
in action command 'lfs'
2017/02/17 18:18:58 [2499/4] params | '--archive'->'--archive' in action 
command 'lfs'
2017/02/17 18:18:58 [2499/4] params | [2] '--archive'->'--archive' in 
action command 'lfs'
2017/02/17 18:18:58 [2499/4] params | '{archive_id}'->'3' in action 
command 'lfs'
2017/02/17 18:18:58 [2499/4] params | [3] '{archive_id}'->'3' in action 
command 'lfs'
2017/02/17 18:18:58 [2499/4] params | '--data'->'--data' in action 
command 'lfs'
2017/02/17 18:18:58 [2499/4] params | [4] '--data'->'--data' in action 
command 'lfs'

2017/02/17 18:18:58 [2499/4] params | '{data}'->'t2' in action command 'lfs'

2017/02/17 18:18:58 [2499/4] params | [5] '{data}'->'t2' in action 
command 'lfs'
2017/02/17 18:18:58 [2499/4] params | 
'/gdatat1/.lustre/fid/{fid}'->'/gdatat1/.lustre/fid/0x200001b70:0x14d:0x0' 
in action command 'lfs'
2017/02/17 18:18:58 [2499/4] params | [6] 
'/gdatat1/.lustre/fid/{fid}'->'/gdatat1/.lustre/fid/0x200001b70:0x14d:0x0' 
in action command 'lfs'
2017/02/17 18:18:58 [2499/4] lhsm_archive | [0x200001b70:0x14d:0x0]: 
action: cmd(lfs hsm_archive --archive 3 --data t2 
/gdatat1/.lustre/fid/0x200001b70:0x14d:0x0)

2017/02/17 18:18:58 [2499/4] ExecCmd | Spawning external command "lfs"
2017/02/17 18:18:58 [2499/4] ExecCmd | Child 2512 terminated with 0

2017/02/17 18:18:58 [2499/4] lhsm_archive | lhsm_archive success for 
'/gdatat1/siteAB/test/resolv.conf', matching rule 'archive_siteAB' 
(fileset=siteAB), last_mod 9.5mi

n ago, size=71


It really looks like it is not considering any other file classes, even 
if parameters are defined in the file classes for lhsm_archive


Gerald





I did also test a config where I did not set a default and I saw that 
the {data} parameter was not set at all and the lfs hsm_archive 
produced an error.

I think this looks like a bug...


I have not found a way to implement that logic any other way, but I 
would like to hear if there is a different approach.



Thanks,
Gerald







------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
robinhood-support mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/robinhood-support






















					Reply via email to