Re: [lustre-discuss] import_set_state_nolock() with binary args in lctl debug_file output?

[Andreas Dilger]

On Aug 14, 2020, at 3:44 PM, Sternberg, Michael G.  wrote:
> 
> 
> In lctl debug_file output, for import_set_state_nolock(), I sometimes see 
> binary arguments (sample snippet at end of post), and figure that's not a 
> good sign. How can I get to the bottom of this?
> 
> The only direct reference I could find is LU-7339 from 2015, which got no 
> replies.

> import_set_state_nolock()) 8ae6833d6000 
> 0<8A>00o<8A>: changing import state 
> from FULL to RECOVER

According to the comments in that ticket, this is a case of the debug message
interpreting the structure incorrectly on the server, since the message is
normally printed on the client:

if (imp->imp_state != LUSTRE_IMP_NEW) {
CDEBUG(D_HA, "%p %s: changing import state from %s to %s\n",
   imp, obd2cli_tgt(imp->imp_obd),
   ptlrpc_import_state_name(imp->imp_state),
   ptlrpc_import_state_name(state));
}

This message could probably be changed from using obd2cli_tgt() to using
imp->imp_obd->obd_name instead.  That said, this is probably a red herring
and totally unrelated to your problem, but a patch against LU-7339 would be
welcome in any case.


Cheers, Andreas







signature.asc
Description: Message signed with OpenPGP
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

[lustre-discuss] Bulk Attach/Detach - Lustre PCC (Persistent Client Cache)

[Pinkesh Valdria]

I am new to Lustre PCC (Persistent Client Cache)  feature.   I was looking at 
the Lustre documentation PCC section and found how I can attach or detach a 
file from PCC.   

http://doc.lustre.org/lustre_manual.xhtml#pcc.operations.detach

 

 

Question:  Is there a command to specify a folder in the command and all files 
under that folder & sub-folders will be attached or detached?   

 

How is everyone doing bulk attach or detach ?    Are folks using some custom 
script to traverse a directory tree (recursively) and for each file found,  
call this command “lfs pcc detach ”  or there is another 
command and I missed it in the docs. 

 

 

 

Thanks, 

Pinkesh Valdria

 

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] status of HSM copytools?

[Simon Guilbault]

Hi

The Compute Canada S3/Ceph copytool was never deployed in production since
S3 backed storage was never installed.
We moved to a tape based system with TSM instead and have been running that
copytool on 3 different sites for the past year.
https://github.com/guilbaults/ct_tsm

On Sun, Aug 23, 2020 at 9:34 AM Matt Rásó-Barnett 
wrote:

> Hi Andrew,
>
> We are using Lemur in production at Cambridge with our Lustre HSM setup,
> although in our case we are using the POSIX backend of it rather than
> the S3 backend.
>
> We adopted it early at the time it was being developed under Intel, as
> the prospect of commercial support (we were Intel IEEL customers at the
> time), and some of the promising features that were being worked on with
> it (snapshot versions, checksums, performance), was very enticing.
>
> But as you've seen, it was a casualty of Intel dropping Lustre, so it's
> not being developed any more. We unfortunately don't have the skills or
> resources internally to develop it ourselves, so I personally am looking
> at how we will transition away from it in the coming months as we
> refresh our HSM setup. I may switch back to using the robinhood
> developed lhsmtool_cmd with our POSIX backends for example.
>
> Lemur is largely doing the job for us however, so it reached a decent
> stage of development before Intel dropped it, but I still wouldn't
> recommend starting with it without someone backing maintenance of it.
>
>  From speaking with the Amazon representative at last year's LUG, I
> understood that they are using Lemur internally for their HSM
> integration as well. I had meant to follow up with them to see if they
> would consider open-sourcing their fork of the project as to my
> knowledge there aren't any other major users of Lemur.
>
> I'd also be interested as you are in hearing what others are using,
> commericial or otherwise. I've never had the feeling that HSM is
> particularly widely used so it'd be good to share information for those
> of us who are!
>
> Best wishes,
>
> Matt
>
> On Sat, Aug 22, 2020 at 10:54:23PM +1000, Andrew Elwell wrote:
> >Hi folks,
> >I'm looking round to see what's current / 'supported' / working in the
> >state of copytools. Ideally one that can migrate to/from object stores
> >(Ceph or S3). The github repo for Lemur
> >(https://github.com/whamcloud/lemur/commits/master) doesn't seem to
> >have had any substantial work since it left Intel - unlucky timing
> >with the owner shift?
> >I've seen another from Compute Canada
> >(https://github.com/ComputeCanada/lustre-obj-copytool) but that too
> >hasn't been touched for years.
> >
> >Anyone care to comment on some working ones? Horror stories? Ones to
> avoid?
> >hey, I'm even (I'll probably regret this) open to _email_ from
> >salesdroids if you have a working product and can point me to some
> >users (but don't try and phone me or make me sit through a webinar).
> >
> >Many thanks
> >
> >
> >Andrew
> >___
> >lustre-discuss mailing list
> >lustre-discuss@lists.lustre.org
> >http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org