Re: [openstack-dev] [MagnetoDB] Bulk load API draft

2014-05-28 Thread Illia Khudoshyn 
Hi Ilya

As for 'string' vs 'number', in MDB REST API we pass numers as strings
since we want to support big ints, so I just wanted to be conform.

As for the last parameter name, I'd prefer 'failed_items', 'cos we 1)
already have 'failed' and I think it would be good if they match 2) they
were actually processed but failed


On Wed, May 28, 2014 at 1:03 PM, Ilya Sviridov wrote:

> Hello Illia,
>
> Great job!
>
> Several moments about spec itself.
>
> "read": "string",
> "processed": "string",
> "failed": "string",
>
> I believe all of them are integers, but not strings.
>
> About Dmitri's suggestion to rename "errors", I would also prefer to do it
> as far as error itself goes below with item.
> What do you think about  "unprocessed_items"?
>
> And I think we can go with it.
>
> Ilya
>
>
>
> On Wed, May 28, 2014 at 10:28 AM, Illia Khudoshyn  > wrote:
>
>> Hi Dima,
>>
>> Sounds good, thank you for the point.
>>
>>
>> On Tue, May 27, 2014 at 7:34 PM, Dmitriy Ukhlov wrote:
>>
>>> Hi Illia,
>>>
>>> Looks good. But I suggest to return all of these fields for positive
>>> request as well as for error request:
>>>
>>> "read": "string",
>>> "processed": "string",
>>> "failed": "string",
>>>
>>> but leave next fields optional and fill them in case of error response
>>> ("failed" > 0) to specify what exactly was happened:
>>>
>>> "last_read":
>>> "errors" (maybe not processed will be better)
>>>
>>>
>>>
>>> On Tue, May 27, 2014 at 3:39 PM, Illia Khudoshyn <
>>> ikhudos...@mirantis.com> wrote:
>>>
 Hi openstackers,

 While working on bulk load, I found previously proposed batch-oriented
 asynchronous approach both resource consuming on server side and somewhat
 complicated to use.
 So I tried to outline some more straightforward streaming way of
 uploading data.

 By the link below you can found a draft for a new streaming API
 https://wiki.openstack.org/wiki/MagnetoDB/streamingbulkload.

 Any feedback is welcome as usual.



 On Wed, May 14, 2014 at 5:04 PM, Illia Khudoshyn <
 ikhudos...@mirantis.com> wrote:

> Hi openstackers,
>
> I'm working on bulk load for MagnetoDB, the facility for inserting
> large amounts of data, like,  millions of rows, gigabytes of data. Below 
> is
> the link to draft API description.
>
>
> https://wiki.openstack.org/wiki/MagnetoDB/bulkload#.5BDraft.5D_MagnetoDB_Bulk_Load_workflow_and_API
>
> Any feedback is welcome.
>
> --
>
> Best regards,
>
> Illia Khudoshyn,
> Software Engineer, Mirantis, Inc.
>
>
>
> 38, Lenina ave. Kharkov, Ukraine
>
> www.mirantis.com 
>
> www.mirantis.ru
>
>
>
> Skype: gluke_work
>
> ikhudos...@mirantis.com
>



 --

 Best regards,

 Illia Khudoshyn,
 Software Engineer, Mirantis, Inc.



 38, Lenina ave. Kharkov, Ukraine

 www.mirantis.com 

 www.mirantis.ru



 Skype: gluke_work

 ikhudos...@mirantis.com

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


>>>
>>>
>>> --
>>> Best regards,
>>> Dmitriy Ukhlov
>>> Mirantis Inc.
>>>
>>> ___
>>> OpenStack-dev mailing list
>>> OpenStack-dev@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>>
>> Best regards,
>>
>> Illia Khudoshyn,
>> Software Engineer, Mirantis, Inc.
>>
>>
>>
>> 38, Lenina ave. Kharkov, Ukraine
>>
>> www.mirantis.com 
>>
>> www.mirantis.ru
>>
>>
>>
>> Skype: gluke_work
>>
>> ikhudos...@mirantis.com
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 

Best regards,

Illia Khudoshyn,
Software Engineer, Mirantis, Inc.



38, Lenina ave. Kharkov, Ukraine

www.mirantis.com 

www.mirantis.ru



Skype: gluke_work

ikhudos...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [MagnetoDB] Bulk load API draft

2014-05-28 Thread Ilya Sviridov 
Hello Illia,

Great job!

Several moments about spec itself.

"read": "string",
"processed": "string",
"failed": "string",

I believe all of them are integers, but not strings.

About Dmitri's suggestion to rename "errors", I would also prefer to do it
as far as error itself goes below with item.
What do you think about  "unprocessed_items"?

And I think we can go with it.

Ilya



On Wed, May 28, 2014 at 10:28 AM, Illia Khudoshyn
wrote:

> Hi Dima,
>
> Sounds good, thank you for the point.
>
>
> On Tue, May 27, 2014 at 7:34 PM, Dmitriy Ukhlov wrote:
>
>> Hi Illia,
>>
>> Looks good. But I suggest to return all of these fields for positive
>> request as well as for error request:
>>
>> "read": "string",
>> "processed": "string",
>> "failed": "string",
>>
>> but leave next fields optional and fill them in case of error response
>> ("failed" > 0) to specify what exactly was happened:
>>
>> "last_read":
>> "errors" (maybe not processed will be better)
>>
>>
>>
>> On Tue, May 27, 2014 at 3:39 PM, Illia Khudoshyn > > wrote:
>>
>>> Hi openstackers,
>>>
>>> While working on bulk load, I found previously proposed batch-oriented
>>> asynchronous approach both resource consuming on server side and somewhat
>>> complicated to use.
>>> So I tried to outline some more straightforward streaming way of
>>> uploading data.
>>>
>>> By the link below you can found a draft for a new streaming API
>>> https://wiki.openstack.org/wiki/MagnetoDB/streamingbulkload.
>>>
>>> Any feedback is welcome as usual.
>>>
>>>
>>>
>>> On Wed, May 14, 2014 at 5:04 PM, Illia Khudoshyn <
>>> ikhudos...@mirantis.com> wrote:
>>>
 Hi openstackers,

 I'm working on bulk load for MagnetoDB, the facility for inserting
 large amounts of data, like,  millions of rows, gigabytes of data. Below is
 the link to draft API description.


 https://wiki.openstack.org/wiki/MagnetoDB/bulkload#.5BDraft.5D_MagnetoDB_Bulk_Load_workflow_and_API

 Any feedback is welcome.

 --

 Best regards,

 Illia Khudoshyn,
 Software Engineer, Mirantis, Inc.



 38, Lenina ave. Kharkov, Ukraine

 www.mirantis.com 

 www.mirantis.ru



 Skype: gluke_work

 ikhudos...@mirantis.com

>>>
>>>
>>>
>>> --
>>>
>>> Best regards,
>>>
>>> Illia Khudoshyn,
>>> Software Engineer, Mirantis, Inc.
>>>
>>>
>>>
>>> 38, Lenina ave. Kharkov, Ukraine
>>>
>>> www.mirantis.com 
>>>
>>> www.mirantis.ru
>>>
>>>
>>>
>>> Skype: gluke_work
>>>
>>> ikhudos...@mirantis.com
>>>
>>> ___
>>> OpenStack-dev mailing list
>>> OpenStack-dev@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> Best regards,
>> Dmitriy Ukhlov
>> Mirantis Inc.
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
>
> Best regards,
>
> Illia Khudoshyn,
> Software Engineer, Mirantis, Inc.
>
>
>
> 38, Lenina ave. Kharkov, Ukraine
>
> www.mirantis.com 
>
> www.mirantis.ru
>
>
>
> Skype: gluke_work
>
> ikhudos...@mirantis.com
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [MagnetoDB] Bulk load API draft

2014-05-28 Thread Illia Khudoshyn 
Hi Dima,

Sounds good, thank you for the point.


On Tue, May 27, 2014 at 7:34 PM, Dmitriy Ukhlov wrote:

> Hi Illia,
>
> Looks good. But I suggest to return all of these fields for positive
> request as well as for error request:
>
> "read": "string",
> "processed": "string",
> "failed": "string",
>
> but leave next fields optional and fill them in case of error response
> ("failed" > 0) to specify what exactly was happened:
>
> "last_read":
> "errors" (maybe not processed will be better)
>
>
>
> On Tue, May 27, 2014 at 3:39 PM, Illia Khudoshyn 
> wrote:
>
>> Hi openstackers,
>>
>> While working on bulk load, I found previously proposed batch-oriented
>> asynchronous approach both resource consuming on server side and somewhat
>> complicated to use.
>> So I tried to outline some more straightforward streaming way of
>> uploading data.
>>
>> By the link below you can found a draft for a new streaming API
>> https://wiki.openstack.org/wiki/MagnetoDB/streamingbulkload.
>>
>> Any feedback is welcome as usual.
>>
>>
>>
>> On Wed, May 14, 2014 at 5:04 PM, Illia Khudoshyn > > wrote:
>>
>>> Hi openstackers,
>>>
>>> I'm working on bulk load for MagnetoDB, the facility for inserting large
>>> amounts of data, like,  millions of rows, gigabytes of data. Below is the
>>> link to draft API description.
>>>
>>>
>>> https://wiki.openstack.org/wiki/MagnetoDB/bulkload#.5BDraft.5D_MagnetoDB_Bulk_Load_workflow_and_API
>>>
>>> Any feedback is welcome.
>>>
>>> --
>>>
>>> Best regards,
>>>
>>> Illia Khudoshyn,
>>> Software Engineer, Mirantis, Inc.
>>>
>>>
>>>
>>> 38, Lenina ave. Kharkov, Ukraine
>>>
>>> www.mirantis.com 
>>>
>>> www.mirantis.ru
>>>
>>>
>>>
>>> Skype: gluke_work
>>>
>>> ikhudos...@mirantis.com
>>>
>>
>>
>>
>> --
>>
>> Best regards,
>>
>> Illia Khudoshyn,
>> Software Engineer, Mirantis, Inc.
>>
>>
>>
>> 38, Lenina ave. Kharkov, Ukraine
>>
>> www.mirantis.com 
>>
>> www.mirantis.ru
>>
>>
>>
>> Skype: gluke_work
>>
>> ikhudos...@mirantis.com
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Best regards,
> Dmitriy Ukhlov
> Mirantis Inc.
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 

Best regards,

Illia Khudoshyn,
Software Engineer, Mirantis, Inc.



38, Lenina ave. Kharkov, Ukraine

www.mirantis.com 

www.mirantis.ru



Skype: gluke_work

ikhudos...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [MagnetoDB] Bulk load API draft

2014-05-27 Thread Dmitriy Ukhlov 
Hi Illia,

Looks good. But I suggest to return all of these fields for positive
request as well as for error request:

"read": "string",
"processed": "string",
"failed": "string",

but leave next fields optional and fill them in case of error response
("failed" > 0) to specify what exactly was happened:

"last_read":
"errors" (maybe not processed will be better)



On Tue, May 27, 2014 at 3:39 PM, Illia Khudoshyn wrote:

> Hi openstackers,
>
> While working on bulk load, I found previously proposed batch-oriented
> asynchronous approach both resource consuming on server side and somewhat
> complicated to use.
> So I tried to outline some more straightforward streaming way of uploading
> data.
>
> By the link below you can found a draft for a new streaming API
> https://wiki.openstack.org/wiki/MagnetoDB/streamingbulkload.
>
> Any feedback is welcome as usual.
>
>
>
> On Wed, May 14, 2014 at 5:04 PM, Illia Khudoshyn 
> wrote:
>
>> Hi openstackers,
>>
>> I'm working on bulk load for MagnetoDB, the facility for inserting large
>> amounts of data, like,  millions of rows, gigabytes of data. Below is the
>> link to draft API description.
>>
>>
>> https://wiki.openstack.org/wiki/MagnetoDB/bulkload#.5BDraft.5D_MagnetoDB_Bulk_Load_workflow_and_API
>>
>> Any feedback is welcome.
>>
>> --
>>
>> Best regards,
>>
>> Illia Khudoshyn,
>> Software Engineer, Mirantis, Inc.
>>
>>
>>
>> 38, Lenina ave. Kharkov, Ukraine
>>
>> www.mirantis.com 
>>
>> www.mirantis.ru
>>
>>
>>
>> Skype: gluke_work
>>
>> ikhudos...@mirantis.com
>>
>
>
>
> --
>
> Best regards,
>
> Illia Khudoshyn,
> Software Engineer, Mirantis, Inc.
>
>
>
> 38, Lenina ave. Kharkov, Ukraine
>
> www.mirantis.com 
>
> www.mirantis.ru
>
>
>
> Skype: gluke_work
>
> ikhudos...@mirantis.com
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Best regards,
Dmitriy Ukhlov
Mirantis Inc.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [MagnetoDB] Bulk load API draft

2014-05-27 Thread Illia Khudoshyn 
Hi openstackers,

While working on bulk load, I found previously proposed batch-oriented
asynchronous approach both resource consuming on server side and somewhat
complicated to use.
So I tried to outline some more straightforward streaming way of uploading
data.

By the link below you can found a draft for a new streaming API
https://wiki.openstack.org/wiki/MagnetoDB/streamingbulkload.

Any feedback is welcome as usual.



On Wed, May 14, 2014 at 5:04 PM, Illia Khudoshyn wrote:

> Hi openstackers,
>
> I'm working on bulk load for MagnetoDB, the facility for inserting large
> amounts of data, like,  millions of rows, gigabytes of data. Below is the
> link to draft API description.
>
>
> https://wiki.openstack.org/wiki/MagnetoDB/bulkload#.5BDraft.5D_MagnetoDB_Bulk_Load_workflow_and_API
>
> Any feedback is welcome.
>
> --
>
> Best regards,
>
> Illia Khudoshyn,
> Software Engineer, Mirantis, Inc.
>
>
>
> 38, Lenina ave. Kharkov, Ukraine
>
> www.mirantis.com 
>
> www.mirantis.ru
>
>
>
> Skype: gluke_work
>
> ikhudos...@mirantis.com
>



-- 

Best regards,

Illia Khudoshyn,
Software Engineer, Mirantis, Inc.



38, Lenina ave. Kharkov, Ukraine

www.mirantis.com 

www.mirantis.ru



Skype: gluke_work

ikhudos...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [MagnetoDB] Bulk load API draft

2014-05-14 Thread Illia Khudoshyn 
Hi openstackers,

I'm working on bulk load for MagnetoDB, the facility for inserting large
amounts of data, like,  millions of rows, gigabytes of data. Below is the
link to draft API description.

https://wiki.openstack.org/wiki/MagnetoDB/bulkload#.5BDraft.5D_MagnetoDB_Bulk_Load_workflow_and_API

Any feedback is welcome.

-- 

Best regards,

Illia Khudoshyn,
Software Engineer, Mirantis, Inc.



38, Lenina ave. Kharkov, Ukraine

www.mirantis.com 

www.mirantis.ru



Skype: gluke_work

ikhudos...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev