Re: [ceph-users] Hardware selection for ceph backup on ceph

Hello Stefan, AMD EPYC > great choice! Has anybody experience with the drives? some of our customers have different toshiba MG06SCA drives and they work great according to them. Can't say for MG07ACA but to be honest, I don't think there should be a huge difference. -- Martin Ve

Re: [ceph-users] Failed to encode map errors

Hello, what versions of Ceph are you running? -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich HRB

Re: [ceph-users] Migrating from block to lvm

hardware performance given. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich HRB 231263 Web: https

Re: [ceph-users] Updating crush location on all nodes of a cluster

) > Maybe the PG UPMAP is something that you can use for this, but your cluster hardware and configuration should always be configured to handle workloads like this rebalance without impacting your clients. See 1. 4. Something else? > Thank you for your time and your help. :) > You

Re: [ceph-users] Can't create erasure coded pools with k+m greater than hosts?

, PSU failure, ...) causes you and all attached clients, especially bad with VMs on that Ceph cluster, to stop any IO or even crash completely. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h,

Re: [ceph-users] rgw S3 lifecycle cannot keep up

Hello Christian, the problem is, that HDD is not capable of providing lots of IOs required for "~4 million small files". -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247

Re: [ceph-users] Need advice with setup planning

github. > Thanks for the feedback, we do need to update the videos to the newest version but our time is unfortunately limited :/ -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich

Re: [ceph-users] Need advice with setup planning

how or on > which servers. > Just put it on the servers as well, with croit it is just a click away and everything is included in our interface. If not using croit, you can still install it on the same systems and configure it by hand/script. -- Martin Verges Managing director Mobile: +49

Re: [ceph-users] Debian Buster builds

-mirror. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich HRB 231263 Web: https://croit.io YouTube: https

Re: [ceph-users] Two questions about ceph update/upgrade strategies

this video https://www.youtube.com/watch?v=Jrnzlylidjs. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich

Re: [ceph-users] bluestore block.db on SSD, where block.wal?

Hello, please see https://www.mail-archive.com/ceph-users@lists.ceph.com/msg54607.html and http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-October/030740.html . -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges

Re: [ceph-users] Using Ceph Ansible to Add Nodes to Cluster at Weight 0

Hello Mike, there is no problem adding 100 OSDs at the same time if your cluster is configured correctly. Just add the OSDs and let the cluster slowly (as fast as your hardware supports without service interruption) rebalance. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail

Re: [ceph-users] performance in a small cluster

Hello Robert, We have identified the performance settings in the BIOS as a major factor > could you share your insights what options you changed to increase performance and could you provide numbers to it? Many thanks in advance -- Martin Verges Managing director Mobile: +49 174 933569

Re: [ceph-users] SSD Sizing for DB/WAL: 4% for large drives?

d use "ceph daemon osd.X perf dump" and look for "db_used_bytes" and "wal_used_bytes", but without guarantee from my side. As far I know, it would be ok to choose values from 2-4% depending on your usage and configuration. -- Martin Verges Managing director Mobile:

Re: [ceph-users] SSD Sizing for DB/WAL: 4% for large drives?

Hello Jake, do you have any latency requirements that you do require the DB/WAL at all? If not, CephFS with EC on SATA HDD works quite well as long as you have the metadata on a separate ssd pool. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat

Re: [ceph-users] Post-mortem analisys?

Hello Marco, first of all, hyperconverged setups with public accessable VMs could be affected by DDoS attacks or other harmful issues that causes cascading errors in your infrastructure. Are you sure your network worked correctly at the time? -- Martin Verges Managing director Mobile: +49 174

Re: [ceph-users] Poor performance for 512b aligned "partial" writes from Windows guests in OpenStack + potential fix

kes a difference in the individual case with virtual disks by the underlying physical disks, I can't say. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verge

Re: [ceph-users] Poor performance for 512b aligned "partial" writes from Windows guests in OpenStack + potential fix

64.rpm.zip 545527fbb28af0c0ff4611fa20be0460 hugo-6.8.0.x86_64.tar.gz.zip -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht

Re: [ceph-users] Poor performance for 512b aligned "partial" writes from Windows guests in OpenStack + potential fix

meone that has bought the wrong disk. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich HRB 231263

Re: [ceph-users] Ceph cluster available to clients with 2 different VLANs ?

Hello, configure a gateway on your router or use a good rack switch that can provide such features and use layer3 routing to connect different vlans / ip zones. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH

Re: [ceph-users] hardware requirements for metadata server

maybe more for bigger deployments. Maybe you can provide some insights about your typical data (size, count,..) and don't forget, you can scale by adding additional online mds as well. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: ht

Re: [ceph-users] Access cephfs from second public network

ther proxy / modification / ... technology to connect as a Ceph client to a Ceph storage within a different IP network. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verge

Re: [ceph-users] v14.2.0 Nautilus released

packages on the official mirrors, please feel free contact us! -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich

Re: [ceph-users] Intel D3-S4610 performance

r the same, and maybe it will be a good choice too. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich

Re: [ceph-users] 3-node cluster with 3 x Intel Optane 900P - very low benchmarked performance (200 IOPS)?

time_based --group_reporting --name=journal-test; done - Please share the results that we know what's possible with your hardware. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Fresenius

Re: [ceph-users] PGs stuck in created state

Hello, try restarting every osd if possible. Upgrade to a recent ceph version. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com

Re: [ceph-users] Ceph REST API

feel free to contact us and we will gladly provide an update within a few days. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com

Re: [ceph-users] Ceph bluestore performance on 4kn vs. 512e?

at once but I'm unsure if that is given in any use case or in a Ceph specific scenario, therefore be save and choose 4Kn drives. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munic

Re: [ceph-users] Bluestore HDD Cluster Advice

ad buy the correct hardware for your use case and configure the cluster accordingly. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com

Re: [ceph-users] Simple API to have cluster healthcheck ?

ything you see in our GUI can be reached through API's. To get a glimpse of the possibilities, look at https://croit.io/screenshots. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 8

Re: [ceph-users] Commercial support

Hello Ketil, we as croit offer commercial support for Ceph as well as our own Ceph based unified storage solution. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin

Re: [ceph-users] MON dedicated hosts

money than required. 2. Maybe think of it when your cluster becomes >500, maybe 1k OSDs or simply if the cluster design would be easier with dedicated servers. Hint: Our training's cover a Ceph cluster planing session that covers exactly such topics. See https://croit.io/training. --

Re: [ceph-users] Journal drive recommendation

Hello, what type of SSD data drives do you plan to use? In general, I would not recommend to use external journal on ssd OSDs, but it is possible to squeeze out a bit more performance depending on your data disks. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver

Re: [ceph-users] Low traffic Ceph cluster with consumer SSD.

happy with. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich HRB 231263 Web: https://croit.io YouTube: https

Re: [ceph-users] Benchmark performance when using SSD as the journal

Please never use the Datasheet values to select your SSD. We never had a single one that that delivers the shown perfomance in a Ceph Journal use case. However, do not use Filestore anymore. Especialy with newer kernel versions. Use Bluestore instead. -- Martin Verges Managing director Mobile

Re: [ceph-users] Luminous or Mimic client on Debian Testing (Buster)

min,key=client.admin ===== -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich HRB 231263 Web: http

Re: [ceph-users] Luminous or Mimic client on Debian Testing (Buster)

X.X.X.X:6789:/ /mnt/ -o name=admin,secret=XXX = -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register

Re: [ceph-users] Supermicro server 5019D8-TR12P for new Ceph cluster

rtin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich HRB 231263 Web: https://croit.io YouTube: https://goo.gl/PGE1Bx

Re: [ceph-users] read performance, separate client CRUSH maps or limit osd read access from each client

Btw. all this could be done with a few simple clicks through our web frontend. Even if you want to export it via CephFS / NFS / .. it is possible to set it on a per folder level. Feel free to take a look at https://www.youtube.com/watch?v=V33f7ipw9d4 to see how easy it could be. -- Martin Verges Man

Re: [ceph-users] read performance, separate client CRUSH maps or limit osd read access from each client

get replicated to the other building. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich HRB 231263 Web

Re: [ceph-users] ceph 12.2.9 release

.youtube.com/watch?v=Jrnzlylidjs to see how we upgrade from Ceph Luminous to Ceph Mimic. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE31063849

Re: [ceph-users] ceph 12.2.9 release

, it just has to run! On the 12.11. on the Ceph Day in Berlin I can give you information directly about it. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT

Re: [ceph-users] Packages for debian in Ceph repo

Hello, we provide a public mirror documented on https://croit.io/2018/09/23/2018-09-23-debian-mirror for Ceph Mimic on Debian Stretch. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247

Re: [ceph-users] Lost machine with MON and MDS

Hello, did you lost the only mon as well? If yes, restore it not easy but possible. The mds is no problem just install it. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO

Re: [ceph-users] Client new version than server?

Hello, In my opinion it is not a problem. It could be a problem on mayor releases (read the release notes to check for incompatibilities) but minor version differences shouldn't be a problem. In the most environments I know are different client versions connecting to a cluster. -- Martin V

Re: [ceph-users] NVME Intel Optane - same servers different performance

searched for possible sources of error. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich HRB 231263 Web

Re: [ceph-users] NVME Intel Optane - same servers different performance

Hello Steven, are you sure that the systems are exact the same? Sometimes vendors place extension cards into different PCIe slots. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247

Re: [ceph-users] Monitor Recovery

ut this? As long as you remaining MONs, you can remove it and add it again. However I would suggest to find out whats wrong with it. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.ver...@croit.io Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 812