[ceph-users] Re: A practical approach to efficiently store 100 billions small objects in Ceph

2021-08-31 Thread Loïc Dachary
Hi, I incorrectly wrote that EOS[0] was using Ceph and RDB. I edited the object storage design[1] to remove that and avoid any confusion. Now I'll try to figure out what led me to make this significant mistake. My apologies for misrepresenting EOS. Cheers [0]

[ceph-users] Re: A practical approach to efficiently store 100 billions small objects in Ceph

2021-08-30 Thread Loïc Dachary
Bonjour, In the past months benchmarks were written[0] and run in the grid5000 cluster[1] to verify there was no blocker with the proposed approach[2]. The results were published today[3]. In the following weeks the Software Heritage codebase will be modified to implement the design[4].

[ceph-users] Re: A practical approach to efficiently store 100 billions small objects in Ceph

2021-03-11 Thread Szabo, Istvan (Agoda)
Cc: Ceph Users ; swh-de...@inria.fr Subject: [ceph-users] Re: A practical approach to efficiently store 100 billions small objects in Ceph Email received from outside the company. If in doubt don't click links nor open attachments! Hi Konstantin, Thanks for the

[ceph-users] Re: A practical approach to efficiently store 100 billions small objects in Ceph

2021-03-11 Thread Szabo, Istvan (Agoda)
wh-de...@inria.fr > Subject: [ceph-users] Re: A practical approach to efficiently store 100 > billions small objects in Ceph > > Email received from outside the company. If in doubt don't click links nor > open attachments! > > > Hi Konstan

[ceph-users] Re: A practical approach to efficiently store 100 billions small objects in Ceph

2021-03-11 Thread Loïc Dachary
> From: Loïc Dachary >> Sent: Thursday, March 11, 2021 2:13 AM >> To: Konstantin Shalygin >> Cc: Ceph Users ; swh-de...@inria.fr >> Subject: [ceph-users] Re: A practical approach to efficiently store 100 >> billions small objects in Ceph >> >> Email received

[ceph-users] Re: A practical approach to efficiently store 100 billions small objects in Ceph

2021-03-10 Thread Szabo, Istvan (Agoda)
: A practical approach to efficiently store 100 billions small objects in Ceph Email received from outside the company. If in doubt don't click links nor open attachments! Hi Konstantin, Thanks for the advice. Luckily objects are packed together and Ceph will only see

[ceph-users] Re: A practical approach to efficiently store 100 billions small objects in Ceph

2021-03-10 Thread Loïc Dachary
inal Message- > From: Loïc Dachary > Sent: Thursday, March 11, 2021 2:13 AM > To: Konstantin Shalygin > Cc: Ceph Users ; swh-de...@inria.fr > Subject: [ceph-users] Re: A practical approach to efficiently store 100 > billions small objects in Ceph > > Email received from ou

[ceph-users] Re: A practical approach to efficiently store 100 billions small objects in Ceph

2021-03-10 Thread Loïc Dachary
Hi Konstantin, Thanks for the advice. Luckily objects are packed together and Ceph will only see larger objects. The median object size is ~4KB, written in RBD images using the default 4MB[0] object size. That will be ~100 millions RADOS objects instead of 100 billions. Cheers [0]

[ceph-users] Re: A practical approach to efficiently store 100 billions small objects in Ceph

2021-03-10 Thread Marc
Bonjour Loic, I think we should all use Bonjour and Merci more often, that sounds much better than Hi, Thanks and Cheers. I am looking forward to reading your research! > -Original Message- > From: Loïc Dachary > Sent: 10 March 2021 15:54 > To: Ceph Users > Cc: swh-de...@inria.fr >