[ceph-users] [ceph-users]

[Amit Ghadge]

Hi All,

I have question regarding Ceph Nautilus upgrade. In our test environment
upgrading
Luminous to Nautilus 14.2.8, and after enable msgr2, we seen one of the mon
node restarted, my question is this normal process of restart mon service
and 2nd question is, we using below mon_host format that can be correct or
move to v2?

mon_host = 10.44.172.181,10.44.172.182,10.44.172.183


Thanks,
AmitG
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Cephfs IO halt on Node failure

[Eugen Block]

What’s your pool configuration wrt min_size and crush rules?


Zitat von Amudhan P :


Hi,

I am using ceph Nautilus cluster with below configuration.

3 node's (Ubuntu 18.04) each has 12 OSD's, and mds, mon and mgr are running
in shared mode.

The client mounted through ceph kernel client.

I was trying to emulate a node failure when a write and read were going on
(replica2) pool.

I was expecting read and write continue after a small pause due to a Node
failure but it halts and never resumes until the failed node is up.

I remember I tested the same scenario before in ceph mimic where it
continued IO after a small pause.

regards
Amudhan P
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io



___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: What is a pgmap?

[Bryan Henderson]

>I thought it was a method (the method?) to know if a PG comes back from a
>crashed OSD/host, to know if it was up-to-date or old since it would have
>an older timestamp.

Thanks.  That's a reasonable theory.  Maybe I'll look in the code and see if
I can confirm it.

And it means on my cluster, once an hour would probably be sufficient.

>I was sure it was updated exactly once per second.

Because there's an infamous cluster log message every time the pgmap updates,
I know for me it is about 10 times a minute, in a pattern that is neither
periodic nor random.  Maybe once per second is the maximum frequency and it
depends upon how frequently PGs are written to.

-- 
Bryan Henderson   San Jose, California
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Cephfs IO halt on Node failure

[Amudhan P]

Hi,

I am using ceph Nautilus cluster with below configuration.

3 node's (Ubuntu 18.04) each has 12 OSD's, and mds, mon and mgr are running
in shared mode.

The client mounted through ceph kernel client.

I was trying to emulate a node failure when a write and read were going on
(replica2) pool.

I was expecting read and write continue after a small pause due to a Node
failure but it halts and never resumes until the failed node is up.

I remember I tested the same scenario before in ceph mimic where it
continued IO after a small pause.

regards
Amudhan P
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Cephfs - NFS Ganesha

[Amudhan P]

Tried to add it to ganesha.conf but didn't work out.

I tried using the default "ganesh-ceph.conf" file comes which comes with
"ganesha-ceph" installation is working fine.
I will try again using conf file provided in nfs-ganesha github.

On Fri, May 15, 2020 at 6:30 PM Daniel Gryniewicz  wrote:

> It sounds like you're putting the FSAL_CEPH config in another file in
> /etc/ganesha.  Ganesha only loads one file: /etc/ganesha/ganesha.conf -
> other files need to be included in that file with the %include command.
> For a simple config like yours, just use the single
> /etc/ganesha/ganesha.conf file.
>
> Daniel
>
> On 5/15/20 4:59 AM, Amudhan P wrote:
> > Hi Rafael,
> >
> > I have used config you have provided but still i am not able mount nfs. I
> > don't see any error in log msg
> >
> > Output from ganesha.log
> > ---
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8732[main]
> > main :MAIN :EVENT :ganesha.nfsd Starting: Ganesha Version 2.6.0
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > nfs_set_param_from_conf :NFS STARTUP :EVENT :Configuration file
> > successfully parsed
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > init_server_pkgs :NFS STARTUP :EVENT :Initializing ID Mapper.
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > init_server_pkgs :NFS STARTUP :EVENT :ID Mapper successfully initialized.
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > lower_my_caps :NFS STARTUP :EVENT :CAP_SYS_RESOURCE was successfully
> > removed for proper quota
> >   management in FSAL
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > lower_my_caps :NFS STARTUP :EVENT :currenty set capabilities are: =
> > cap_chown,cap_dac_overrid
> >
> e,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_
> >
> raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_time,cap_sys_tty
> >
> _config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap+ep
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > nfs_start_grace :STATE :EVENT :NFS Server Now IN GRACE, duration 90
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > nfs_Init_svc :DISP :CRIT :Cannot acquire credentials for principal nfs
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > nfs_Init_admin_thread :NFS CB :EVENT :Admin thread initialized
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > nfs_rpc_cb_init_ccache :NFS STARTUP :EVENT :Callback creds directory
> > (/var/run/ganesha) alrea
> > dy exists
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > nfs_rpc_cb_init_ccache :NFS STARTUP :WARN
> > :gssd_refresh_krb5_machine_credential failed (-1765
> > 328160:0)
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > nfs_Start_threads :THREAD :EVENT :Starting delayed executor.
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > nfs_Start_threads :THREAD :EVENT :9P/TCP dispatcher thread was started
> > successfully
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> > ganesha.nfsd-8738[_9p_disp] _9p_dispatcher_thread :9P DISP :EVENT :9P
> > dispatcher started
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > nfs_Start_threads :THREAD :EVENT :gsh_dbusthread was started successfully
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > nfs_Start_threads :THREAD :EVENT :admin thread was started successfully
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > nfs_Start_threads :THREAD :EVENT :reaper thread was started successfully
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > nfs_Start_threads :THREAD :EVENT :General fridge was started successfully
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > nfs_start :NFS STARTUP :EVENT
> > :-
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > nfs_start :NFS STARTUP :EVENT : NFS SERVER INITIALIZED
> > 15/05/2020 08:50:43 : epoch 5ebe57e3 : strgcntrl :
> ganesha.nfsd-8738[main]
> > nfs_start :NFS STARTUP :EVENT
> > :-
> > 15/05/2020 08:52:13 : epoch 5ebe57e3 : strgcntrl :
> > ganesha.nfsd-8738[reaper] nfs_lift_grace_locked :STATE :EVENT :NFS Server
> > Now NOT IN GRACE
> >
> > Regards
> > Amudhan P
> >
> > On Fri, May 15, 2020 at 1:01 PM Rafael Lopez