Hi, Openstackers,
I'm currently working on adding bulk data load functionality to MagnetoDB.
This functionality implies inserting huge amounts of data (billions of
rows, gigabytes of data). The data being uploaded is a set of JSON's (for
now). The question I'm interested in is a way of data transp
Hi Iliia,
I would take a look into BSON http://bsonspec.org/
Cheers,
Serge Kovaleff
On Thu, Mar 27, 2014 at 8:23 PM, Illia Khudoshyn wrote:
> Hi, Openstackers,
>
> I'm currently working on adding bulk data load functionality to MagnetoDB.
> This functionality implies inserting huge amounts of d
2014-03-28 11:29 GMT+02:00 Serge Kovaleff :
> Hi Iliia,
>
> I would take a look into BSON http://bsonspec.org/
>
> Cheers,
> Serge Kovaleff
>
> On Thu, Mar 27, 2014 at 8:23 PM, Illia Khudoshyn
> wrote:
>
>> Hi, Openstackers,
>>
>> I'm currently working on adding bulk data load functionality to
>>
Hi guys,
I suggest taking a look, how Swift and Ceph do such things.
2014-03-28 12:33 GMT+02:00 Maksym Iarmak :
>
>
>
> 2014-03-28 11:29 GMT+02:00 Serge Kovaleff :
>
> Hi Iliia,
>>
>> I would take a look into BSON http://bsonspec.org/
>>
>> Cheers,
>> Serge Kovaleff
>>
>> On Thu, Mar 27, 2014 a
Maksym Iarmak wrote:
> I suggest taking a look, how Swift and Ceph do such things.
under swift (and CEPH via the radosgw which implement swift API) we are
using POST and PUT which has been working relatively well
Chmouel
___
OpenStack-dev mailing list
O
On 03/28/2014 11:29 AM, Serge Kovaleff wrote:
Hi Iliia,
I would take a look into BSON http://bsonspec.org/
Cheers,
Serge Kovaleff
On Thu, Mar 27, 2014 at 8:23 PM, Illia Khudoshyn
mailto:ikhudos...@mirantis.com>> wrote:
Hi, Openstackers,
I'm currently working on adding bulk data loa
Dmitriy Ukhlov wrote:
> > I guess if we a talking about cassandra batch loading the fastest way
> is to generate sstables locally and load it into Cassandra via JMX or
> sstableloader
> http://www.datastax.com/dev/blog/bulk-loading
>
>
Good idea, Dmitriy. IMHO bulk load is back-end specific task
Bulk loading with sstableloader is blazingly fast (the price to pay is that's
not portable of course).
Also it's network efficient thanks to SSTable compression. If the network is
not a limiting factor then LZ4 will be great.
Le Vendredi 28 mars 2014 13h46, Aleksandr Chudnovets
a écrit :