date:20180121

Re: [ceph-users] How to set mon-clock-drift-allowed tunable

2018-01-21 Thread Wido den Hollander




On 01/22/2018 08:11 AM, Hüseyin Atatür YILDIRIM wrote:


Hello,

How can I set mon_clock_drift_allowed  tunable;  should  I set it  at 
monitor nodes  /etc/ceph/ceph.conf   or   where ?  In  [global]  
section, or   in a [mon] section ?




Both would work, but the [mon] section is the best.


All these are not clear in the documentation.

After setting, should I restart the daemons, or what specific daemon ? 
Or  should I reboot the node?




Reboot the Monitors one by one, wait a few minutes in between.

But why are you setting this? Because if you have your clocks drifting 
that often you should fix your NTP.


A properly configured machine should never have this configuration 
setting changed.


Wido


Thank you,

Atatur



  
**Hüseyin Atatür YILDIRIM
SİSTEM MÜHENDİSİ
Üniversiteler Mah. İhsan Doğramacı Bul. ODTÜ Teknokent Havelsan A.Ş. 
23/B Çankaya Ankara TÜRKİYE

+90 312 292 74 00   +90 312 219 57 97

YASAL UYARI: Bu elektronik posta işbu linki kullanarak ulaşabileceğiniz 
Koşul ve Şartlar dokümanına tabidir. 

LEGAL NOTICE: This e-mail is subject to the Terms and Conditions 
document which can be accessed with this link. 

	Lütfen gerekmedikçe bu sayfa ve eklerini yazdırmayınız / Please 
consider the environment before printing this email





___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] How to set mon-clock-drift-allowed tunable

2018-01-21 Thread Hüseyin Atatür YILDIRIM


Hello,

How can I set mon_clock_drift_allowed  tunable;  should  I set it  at monitor 
nodes  /etc/ceph/ceph.conf   or   where ?  In  [global]  section, or   in a 
[mon] section ?
All these are not clear in the documentation.

After setting, should I restart the daemons, or what specific daemon ? Or  
should I reboot the node?


Thank you,
Atatur




[cid:imaged51036.PNG@3427f5e9.4f8ee361] 
[cid:image9e5230.JPG@7be42bc5.49865965]
Hüseyin Atatür YILDIRIM
SİSTEM MÜHENDİSİ
Üniversiteler Mah. İhsan Doğramacı Bul. ODTÜ Teknokent Havelsan A.Ş. 23/B 
Çankaya Ankara TÜRKİYE
[cid:image4739b6.PNG@4833e2cb.4e884e1f] +90 312 292 74 00   
[cid:image6a41d4.PNG@7a7e7b1a.4caaa25e] +90 312 219 57 97


[cid:image4291ae.JPG@37fdf40b.419fe600]
YASAL UYARI: Bu elektronik posta işbu linki kullanarak ulaşabileceğiniz Koşul 
ve Şartlar dokümanına tabidir. 

LEGAL NOTICE: This e-mail is subject to the Terms and Conditions document which 
can be accessed with this link. 


[http://www.havelsan.com.tr/Library/images/mail/email.jpg]  Lütfen 
gerekmedikçe bu sayfa ve eklerini yazdırmayınız / Please consider the 
environment before printing this email


___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] What is the should be the expected latency of 10Gbit network connections

2018-01-21 Thread Wido den Hollander




On 01/20/2018 02:02 PM, Marc Roos wrote:
  
If I test my connections with sockperf via a 1Gbit switch I get around

25usec, when I test the 10Gbit connection via the switch I have around
12usec is that normal? Or should there be a differnce of 10x.



No, that's normal.

Tests with 8k ping packets over different links I did:

1GbE:  0.800ms
10GbE: 0.200ms
40GbE: 0.150ms

Wido


sockperf ping-pong

sockperf: Warmup stage (sending a few dummy messages)...
sockperf: Starting test...
sockperf: Test end (interrupted by timer)
sockperf: Test ended
sockperf: [Total Run] RunTime=10.100 sec; SentMessages=432875;
ReceivedMessages=432874
sockperf: = Printing statistics for Server No: 0
sockperf: [Valid Duration] RunTime=10.000 sec; SentMessages=428640;
ReceivedMessages=428640
sockperf: > avg-lat= 11.609 (std-dev=1.684)
sockperf: # dropped messages = 0; # duplicated messages = 0; #
out-of-order messages = 0
sockperf: Summary: Latency is 11.609 usec
sockperf: Total 428640 observations; each percentile contains 4286.40
observations
sockperf: --->  observation =  856.944
sockperf: ---> percentile  99.99 =   39.789
sockperf: ---> percentile  99.90 =   20.550
sockperf: ---> percentile  99.50 =   17.094
sockperf: ---> percentile  99.00 =   15.578
sockperf: ---> percentile  95.00 =   12.838
sockperf: ---> percentile  90.00 =   12.299
sockperf: ---> percentile  75.00 =   11.844
sockperf: ---> percentile  50.00 =   11.409
sockperf: ---> percentile  25.00 =   11.124
sockperf: --->  observation =8.888

sockperf: Warmup stage (sending a few dummy messages)...
sockperf: Starting test...
sockperf: Test end (interrupted by timer)
sockperf: Test ended
sockperf: [Total Run] RunTime=1.100 sec; SentMessages=22065;
ReceivedMessages=22064
sockperf: = Printing statistics for Server No: 0
sockperf: [Valid Duration] RunTime=1.000 sec; SentMessages=20056;
ReceivedMessages=20056
sockperf: > avg-lat= 24.861 (std-dev=1.774)
sockperf: # dropped messages = 0; # duplicated messages = 0; #
out-of-order messages = 0
sockperf: Summary: Latency is 24.861 usec
sockperf: Total 20056 observations; each percentile contains 200.56
observations
sockperf: --->  observation =   77.158
sockperf: ---> percentile  99.99 =   54.285
sockperf: ---> percentile  99.90 =   37.864
sockperf: ---> percentile  99.50 =   34.406
sockperf: ---> percentile  99.00 =   33.337
sockperf: ---> percentile  95.00 =   27.497
sockperf: ---> percentile  90.00 =   26.072
sockperf: ---> percentile  75.00 =   24.618
sockperf: ---> percentile  50.00 =   24.443
sockperf: ---> percentile  25.00 =   24.361
sockperf: --->  observation =   16.746
[root@c01 sbin]# sockperf ping-pong -i 192.168.0.12 -p 5001 -t 10
sockperf: == version #2.6 ==
sockperf[CLIENT] send on:sockperf: using recvfrom() to block on
socket(s)








___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] udev rule or script to auto add bcache devices?

2018-01-21 Thread Wido den Hollander




On 01/20/2018 07:56 PM, Stefan Priebe - Profihost AG wrote:

Hello,

bcache didn't supported partitions on the past so that a lot of our osds
have their data directly on:
/dev/bcache[0-9]

But that means i can't give them the needed part type of
4fbd7e29-9d25-41b8-afd0-062c0ceff05d and that means that the activation
with udev und ceph-disk does not work.

Had anybody already fixed this or hacked something together?


Not really. But with ceph-volume around the corner, isn't that something 
that might work? It doesn't use udev anymore.


You need to run Luminous though.

Wido



Greets,
Stefan
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Ubuntu 17.10 or Debian 9.3 + Luminous = random OS hang ?

2018-01-21 Thread Youzhong Yang

Thanks. I applied the workaround to .vmx and rebooted all VMs. No more
freeze!

On Sun, Jan 21, 2018 at 3:43 PM, Nick Fisk  wrote:

> How up to date is your VM environment? We saw something very similar last
> year with Linux VM’s running newish kernels. It turns out newer kernels
> supported a new feature of the vmxnet3 adapters which had a bug in ESXi.
> The fix was release last year some time in ESXi6.5 U1, or a workaround was
> to set an option in the VM config.
>
>
>
> https://kb.vmware.com/s/article/2151480
>
>
>
>
>
>
>
> *From:* ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On Behalf
> Of *Youzhong Yang
> *Sent:* 21 January 2018 19:50
> *To:* Brad Hubbard 
> *Cc:* ceph-users 
> *Subject:* Re: [ceph-users] Ubuntu 17.10 or Debian 9.3 + Luminous =
> random OS hang ?
>
>
>
> As someone suggested, I installed linux-generic-hwe-16.04 package on
> Ubuntu 16.04 to get kernel of 17.10, and then rebooted all VMs, here is
> what I observed:
>
> - ceph monitor node froze upon reboot, in another case froze after a few
> minutes
>
> - ceph OSD hosts easily froze
>
> - ceph admin node (which runs no ceph service but ceph-deploy) never
> freezes
>
> - ceph rgw nodes and ceph mgr so far so good
>
>
>
> Here are two images I captured:
>
>
>
> https://drive.google.com/file/d/11hMJwhCF6Tj8LD3nlpokG0CB_
> oZqI506/view?usp=sharing
>
> https://drive.google.com/file/d/1tzDQ3DYTnfDHh_
> hTQb0ISZZ4WZdRxHLv/view?usp=sharing
>
>
>
> Thanks.
>
>
>
> On Sat, Jan 20, 2018 at 7:03 PM, Brad Hubbard  wrote:
>
> On Fri, Jan 19, 2018 at 11:54 PM, Youzhong Yang 
> wrote:
> > I don't think it's hardware issue. All the hosts are VMs. By the way,
> using
> > the same set of VMWare hypervisors, I switched back to Ubuntu 16.04 last
> > night, so far so good, no freeze.
>
> Too little information to make any sort of assessment I'm afraid but,
> at this stage, this doesn't sound like a ceph issue.
>
>
> >
> > On Fri, Jan 19, 2018 at 8:50 AM, Daniel Baumann 
> > wrote:
> >>
> >> Hi,
> >>
> >> On 01/19/18 14:46, Youzhong Yang wrote:
> >> > Just wondering if anyone has seen the same issue, or it's just me.
> >>
> >> we're using debian with our own backported kernels and ceph, works rock
> >> solid.
> >>
> >> what you're describing sounds more like hardware issues to me. if you
> >> don't fully "trust"/have confidence in your hardware (and your logs
> >> don't reveal anything), I'd recommend running some burn-in tests
> >> (memtest, cpuburn, etc.) on them for 24 hours/machine to rule out
> >> cpu/ram/etc. issues.
> >>
> >> Regards,
> >> Daniel
> >> ___
> >> ceph-users mailing list
> >> ceph-users@lists.ceph.com
> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
> >
> >
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
>
>
> --
> Cheers,
> Brad
>
>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Fwd: Ceph team involvement in Rook (Deploying Ceph in Kubernetes)

2018-01-21 Thread Hans van den Bogert

Should I summarize this is ceph-helm being being EOL? If I'm spinning up a
toy cluster for a homelab, should I invest time in Rook, or stay with
ceph-helm for now?

On Fri, Jan 19, 2018 at 11:55 AM, Kai Wagner  wrote:

> Just for those of you who are not subscribed to ceph-users.
>
>
>  Forwarded Message 
> Subject: Ceph team involvement in Rook (Deploying Ceph in Kubernetes)
> Date: Fri, 19 Jan 2018 11:49:05 +0100
> From: Sebastien Han  
> To: ceph-users  ,
> Squid Cybernetic  ,
> Dan Mick  , Chen, Huamin
>  , John Spray 
> , Sage Weil  ,
> bas...@tabbara.com
>
> Everyone,
>
> Kubernetes is getting bigger and bigger. It has become the platform of
> choice to run microservices applications in containers, just like
> OpenStack did for and Cloud applications in virtual machines.
>
> When it comes to container storage there are three key aspects:
>
> * Providing persistent storage to containers, Ceph has drivers in
> Kuberntes already with kRBD and CephFS
> * Containerizing the storage itself, so efficiently running Ceph
> services in Containers. Currently, we have ceph-container
> (https://github.com/ceph/ceph-container)
> * Deploying the containerized storage in Kubernetes, we wrote
> ceph-helm charts (https://github.com/ceph/ceph-helm)
>
> The third piece although it's working great has a particular goal and
> doesn't aim to run Ceph just like any other applications in Kuberntes.
> We were also looking for a better abstraction/ease of use for
> end-users, multi-cluster support, operability, life-cycle management,
> centralized operations, to learn more you can 
> readhttp://lists.ceph.com/pipermail/ceph-users-ceph.com/2017-October/021918.html.
> As a consequence, we decided to look at what the ecosystem had to
> offer. As a result, Rook came out, as a pleasant surprise. For those
> who are not familiar with Rook, please visit https://rook.io but in a
> nutshell, Rook is an open source orchestrator for distributed storage
> systems running in cloud-native environments. Under the hood, Rook is
> deploying, operating and managing Ceph life cycle in Kubernetes. Rook
> has a vibrant community and committed developers.
>
> Even if Rook is not perfect (yet), it has firm foundations, and we are
> planning on helping to make it better. We already opened issues for
> that and started doing work with Rook's core developers. We are
> looking at reconciling what is available today
> (rook/ceph-container/helm), reduce the overlap/duplication and all
> work together toward a single and common goal. With this
> collaboration, through Rook, we hope to make Ceph the de facto Open
> Source storage solution for Kubernetes.
>
> These are exciting times, so if you're a user, a developer, or merely
> curious, have a look at Rook and send us feedback!
>
> Thanks!
> --
> Cheers
>
> ––
> Sébastien Han
> Principal Software Engineer, Storage Architect
>
> "Always give 100%. Unless you're giving blood."
>
> Mail: s...@redhat.com
> Address: 11 bis, rue Roquépine - 75008 Paris
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Ubuntu 17.10 or Debian 9.3 + Luminous = random OS hang ?

2018-01-21 Thread Nick Fisk

How up to date is your VM environment? We saw something very similar last year 
with Linux VM’s running newish kernels. It turns out newer kernels supported a 
new feature of the vmxnet3 adapters which had a bug in ESXi. The fix was 
release last year some time in ESXi6.5 U1, or a workaround was to set an option 
in the VM config.

 

https://kb.vmware.com/s/article/2151480

 

 

 

From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of 
Youzhong Yang
Sent: 21 January 2018 19:50
To: Brad Hubbard 
Cc: ceph-users 
Subject: Re: [ceph-users] Ubuntu 17.10 or Debian 9.3 + Luminous = random OS 
hang ?

 

As someone suggested, I installed linux-generic-hwe-16.04 package on Ubuntu 
16.04 to get kernel of 17.10, and then rebooted all VMs, here is what I 
observed:

- ceph monitor node froze upon reboot, in another case froze after a few 
minutes 

- ceph OSD hosts easily froze

- ceph admin node (which runs no ceph service but ceph-deploy) never freezes

- ceph rgw nodes and ceph mgr so far so good

 

Here are two images I captured:

 

https://drive.google.com/file/d/11hMJwhCF6Tj8LD3nlpokG0CB_oZqI506/view?usp=sharing

https://drive.google.com/file/d/1tzDQ3DYTnfDHh_hTQb0ISZZ4WZdRxHLv/view?usp=sharing

 

Thanks.

 

On Sat, Jan 20, 2018 at 7:03 PM, Brad Hubbard mailto:bhubb...@redhat.com> > wrote:

On Fri, Jan 19, 2018 at 11:54 PM, Youzhong Yang mailto:youzh...@gmail.com> > wrote:
> I don't think it's hardware issue. All the hosts are VMs. By the way, using
> the same set of VMWare hypervisors, I switched back to Ubuntu 16.04 last
> night, so far so good, no freeze.

Too little information to make any sort of assessment I'm afraid but,
at this stage, this doesn't sound like a ceph issue.


>
> On Fri, Jan 19, 2018 at 8:50 AM, Daniel Baumann   >
> wrote:
>>
>> Hi,
>>
>> On 01/19/18 14:46, Youzhong Yang wrote:
>> > Just wondering if anyone has seen the same issue, or it's just me.
>>
>> we're using debian with our own backported kernels and ceph, works rock
>> solid.
>>
>> what you're describing sounds more like hardware issues to me. if you
>> don't fully "trust"/have confidence in your hardware (and your logs
>> don't reveal anything), I'd recommend running some burn-in tests
>> (memtest, cpuburn, etc.) on them for 24 hours/machine to rule out
>> cpu/ram/etc. issues.
>>
>> Regards,
>> Daniel
>> ___
>> ceph-users mailing list
>> ceph-users@lists.ceph.com  
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com  
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>




--
Cheers,
Brad

 

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Ubuntu 17.10 or Debian 9.3 + Luminous = random OS hang ?

2018-01-21 Thread Youzhong Yang

As someone suggested, I installed linux-generic-hwe-16.04 package on Ubuntu
16.04 to get kernel of 17.10, and then rebooted all VMs, here is what I
observed:
- ceph monitor node froze upon reboot, in another case froze after a few
minutes
- ceph OSD hosts easily froze
- ceph admin node (which runs no ceph service but ceph-deploy) never freezes
- ceph rgw nodes and ceph mgr so far so good

Here are two images I captured:

https://drive.google.com/file/d/11hMJwhCF6Tj8LD3nlpokG0CB_oZqI506/view?usp=sharing
https://drive.google.com/file/d/1tzDQ3DYTnfDHh_hTQb0ISZZ4WZdRxHLv/view?usp=sharing

Thanks.

On Sat, Jan 20, 2018 at 7:03 PM, Brad Hubbard  wrote:

> On Fri, Jan 19, 2018 at 11:54 PM, Youzhong Yang 
> wrote:
> > I don't think it's hardware issue. All the hosts are VMs. By the way,
> using
> > the same set of VMWare hypervisors, I switched back to Ubuntu 16.04 last
> > night, so far so good, no freeze.
>
> Too little information to make any sort of assessment I'm afraid but,
> at this stage, this doesn't sound like a ceph issue.
>
> >
> > On Fri, Jan 19, 2018 at 8:50 AM, Daniel Baumann 
> > wrote:
> >>
> >> Hi,
> >>
> >> On 01/19/18 14:46, Youzhong Yang wrote:
> >> > Just wondering if anyone has seen the same issue, or it's just me.
> >>
> >> we're using debian with our own backported kernels and ceph, works rock
> >> solid.
> >>
> >> what you're describing sounds more like hardware issues to me. if you
> >> don't fully "trust"/have confidence in your hardware (and your logs
> >> don't reveal anything), I'd recommend running some burn-in tests
> >> (memtest, cpuburn, etc.) on them for 24 hours/machine to rule out
> >> cpu/ram/etc. issues.
> >>
> >> Regards,
> >> Daniel
> >> ___
> >> ceph-users mailing list
> >> ceph-users@lists.ceph.com
> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
> >
> >
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
>
>
>
> --
> Cheers,
> Brad
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] How to set mon-clock-drift-allowed tunable

[ceph-users] How to set mon-clock-drift-allowed tunable

Re: [ceph-users] What is the should be the expected latency of 10Gbit network connections

Re: [ceph-users] udev rule or script to auto add bcache devices?

Re: [ceph-users] Ubuntu 17.10 or Debian 9.3 + Luminous = random OS hang ?

Re: [ceph-users] Fwd: Ceph team involvement in Rook (Deploying Ceph in Kubernetes)

Re: [ceph-users] Ubuntu 17.10 or Debian 9.3 + Luminous = random OS hang ?

Re: [ceph-users] Ubuntu 17.10 or Debian 9.3 + Luminous = random OS hang ?

8 matches

Site Navigation

Mail list logo

Footer information