not research and an answer such as "Read The Fine
Blog/Manual" with a URL would be most welcome :-)
[1] https://wiki.softwareheritage.org/wiki/Preliminary_Object_Storage_Benchmarks
[2] https://docs.ceph.com/en/pacific/rbd/rbd-config-ref/#qos-settings
--
Loïc Dachary, Artisan Logi
-web/
[1]
https://wiki.softwareheritage.org/index.php?title=A_practical_approach_to_efficiently_store_100_billions_small_objects_in_Ceph=revision=1607=1590
On 10/03/2021 15:53, Loïc Dachary wrote:
> Bonjour,
>
> In the past weeks a few mailing list threads[0][1][2] explored th
/A_practical_approach_to_efficiently_store_100_billions_small_objects_in_Ceph
[3] https://wiki.softwareheritage.org/wiki/Premilinary_Object_Storage_Benchmarks
[4] https://forge.softwareheritage.org/T3432
On 10/03/2021 15:53, Loïc Dachary wrote:
> Bonjour,
>
> In the past weeks a few mailing list threads[0][1][2] explored th
may be intercepted, amended,
> lost or deleted, or contain viruses.
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
--
Loïc Dachary, Artisan Logiciel Li
s 1T --data-pool my_pool my_pool_metadata/my_data
>
> ```
>
> First Question: Is this correct?
Yes, as explained here
https://docs.ceph.com/en/latest/rados/operations/erasure-code/#erasure-coding-with-overwrites
I don't know what the qemu equivalent is though.
Cheers
--
Loïc Dachary, Artisan
gt; Agoda Services Co., Ltd.
> e: istvan.sz...@agoda.com
> ---
>
> -Original Message-
> From: Loïc Dachary
> Sent: Thursday, March 11, 2021 2:10 PM
> To: Szabo, Istvan (Agoda)
> Cc: Ceph Users
> Subject: [ceph-user
inal Message-----
> From: Loïc Dachary
> Sent: Thursday, March 11, 2021 2:13 AM
> To: Konstantin Shalygin
> Cc: Ceph Users ; swh-de...@inria.fr
> Subject: [ceph-users] Re: A practical approach to efficiently store 100
> billions small objects in Ceph
>
> Email received from ou
ceph.com/issues/47044
> <https://tracker.ceph.com/issues/47044>
> k
>
> Sent from my iPhone
>
>> On 10 Mar 2021, at 17:55, Loïc Dachary wrote:
>>
>> The next step will be to write and run benchmarks
--
Loïc Dachary, Artisan Logiciel Libre
Op
tps://forge.softwareheritage.org/T3054
[4] https://www.softwareheritage.org/
--
Loïc Dachary, Artisan Logiciel Libre
OpenPGP_signature
Description: OpenPGP digital signature
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to c
ttps://forge.softwareheritage.org/T3064
On 17/02/2021 17:36, Loïc Dachary wrote:
> Bonjour,
>
> TL;DR: Is it more advisable to work on Ceph internals to make it friendly to
> this particular workload or write something similar to EOS[0] (i.e Rocksdb +
> Xrootd + RBD)?
>
> This is a follo
On 18/02/2021 17:55, Dan van der Ster wrote:
> On Thu, Feb 18, 2021 at 12:36 AM Robin H. Johnson wrote:
>> On Wed, Feb 17, 2021 at 05:36:53PM +0100, Loïc Dachary wrote:
>>> Bonjour,
>>>
>>> TL;DR: Is it more advisable to work on Ceph internals to make it
>
Hi Robin,
On 18/02/2021 00:35, Robin H. Johnson wrote:
> On Wed, Feb 17, 2021 at 05:36:53PM +0100, Loïc Dachary wrote:
>> Bonjour,
>>
>> TL;DR: Is it more advisable to work on Ceph internals to make it
>> friendly to this particular workload or write something simila
market.
> After raid you get 1.5PiB per server. This is 30 months of data.
> 2 such servers will hold 5 years of data with minimal problems.
> If you need backup; then buy 2 more sets and just send zfs snapshot
> diffs to this set.
>
>
> On Wed, Feb 17, 2021 at 11:15 PM Loïc Dachar
;
> On Wed, Feb 17, 2021 at 8:14 PM Loïc Dachary wrote:
>> Hi Nathan,
>>
>> Good thinking :-) The names of the objects are indeed the SHA256 of their
>> content, which provides deduplication.
>>
>> Cheers
>>
>> On 17/02/2021 18:04, Nathan Fish wrot
ressed". I don't know if you have many
> duplicate objects in this set, but content-addressing gives you
> object-level dedup for free. Do you have to preserve some meaningful
> object names from the original dataset, or just do you just need some
> kind of ID?
>
> On Wed,
areheritage.org/T3049
[12] https://forge.softwareheritage.org/T3054#58977
--
Loïc Dachary, Artisan Logiciel Libre
OpenPGP_signature
Description: OpenPGP digital signature
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
; [1] https://github.com/ceph/ceph/pull/33434
> [2] https://github.com/ceph/ceph/pull/33365
>
> On 2/14/21 9:21 AM, Loïc Dachary wrote:
>> Bonjour,
>>
>> Reading Karan's blog post about benchmarking the insertion of billions
>> objects to C
On 04/02/2021 12:08, Lionel Bouton wrote:
> Hi,
>
> Le 04/02/2021 à 08:41, Loïc Dachary a écrit :
>> Hi Frederico,
>>
>> On 04/02/2021 05:51, Federico Lucifredi wrote:
>>> Hi Loïc,
>>> I am intrigued, but am missing something: why not using RGW, an
> Federico Lucifredi
> Product Management Director, Ceph Storage Platform
> Red Hat
> A273 4F57 58C0 7FE8 838D 4F87 AEEB EC18 4A73 88AC
> redhat.com <http://redhat.com> TRIED. TESTED. TRUSTED.
>
>
> On Sat, Jan 30, 2021 at 10:01 AM Loïc Dachary <mailto:l...
sGW.
> - Build a small frontend that fetches (with caching) them and provides the
> file contents via whatever your UI is.
>
> On Wed, Feb 3, 2021 at 12:55 AM Burkhard Linke <
> burkhard.li...@computational.bio.uni-giessen.de> wrote:
>
>> Hi,
>>
>> On 2/3/21 9
>
> Just my 2 cents:
>
> You could use the first byte of the SHA sum to identify the image, e.g. using
> a fixed number of 256 images. Or some flexible approach similar to the way
> filestore used to store rados objects.
A friend suggested the same to save space. Good idea.
t-ceph-plugin
https://github.com/ceph-dovecot/dovecot-ceph-plugin/tree/master/src/librmb
Thanks for the pointer.
Cheers
> -Greg
>
> On Mon, Feb 1, 2021 at 12:52 PM Loïc Dachary wrote:
>> Hi Dan,
>>
>> On 01/02/2021 21:13, Dan van der Ster wrote:
>>>
But I did not try and maybe I should.
To be continued.
>
> Cheers, Dan
>
>
> On Sat, Jan 30, 2021, 4:01 PM Loïc Dachary wrote:
>
>> Bonjour,
>>
>> In the context Software Heritage (a noble mission to preserve all source
>> code)[0], artifacts have an average s
o a second site.
>
> --
> Alex Gorbachev
> ISS/Storcium
>
>
>
> On Mon, Feb 1, 2021 at 2:43 AM Loïc Dachary <mailto:l...@dachary.org>> wrote:
>
> Hi Alex,
>
> Using borg would indeed make sense to copy the replicate the rbd con
://docs.softwareheritage.org/
--
Loïc Dachary, Artisan Logiciel Libre
OpenPGP_signature
Description: OpenPGP digital signature
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
25 matches
Mail list logo
