Re: rgw backports to giant update

2015-01-30 Thread Loic Dachary
Hi Yehuda, Thanks a lot for this inventory. I updated the tickets to make sure all of them have the "giant" string in the backport field. I'll try backporting myself which will help when it's trivial. If I hit a conflict I'll get back to you :-) Greetings from Brussels ! On 30/01/2015 20:05, Y

Re: wip-auth

2015-01-30 Thread Sage Weil
Hi Andreas, It looks like that was a stale sha1, but the newer one was also broken. I've retested and it's working for me now. See latest wip-auth, sha1 0c21a7875059bef80842756dfb003f47cc2d66a6. Thanks! sage On Fri, 30 Jan 2015, Andreas Bluemle wrote: > Hi Sage, > > I tried to integrate wi

Re: rgw backports to giant update

2015-01-30 Thread Yehuda Sadeh
Hi, these are the actual issues that I backported: - 9878 rhel7 s3-tests fail due to missing reason - 9918 RGW-Swift: SubUser access permissions, does not seems to workminor - 10103 swift tests failing - 10354 Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi ru

Re: Supporting partial writes for EC backend

2015-01-30 Thread Bassam Tabbara
Thanks, I’ll take a look at osd_internals. Are you saying that a cache tier —> EC Pool is good enough? What if any is the downside to using a cache tier ontop of an EC pool? I assume that the size of cache is going to be important, and so is the migration algorithm between the cache tier and t

Re: Supporting partial writes for EC backend

2015-01-30 Thread Samuel Just
This would need to be a new backend, and it's going to be a massive amount of work. You want to start by reading and understanding all of the docs in osd_internals. I also suspect you'll get a lot of what you need from using a replicated cache in front of an EC pool. -Sam On Fri, Jan 30, 2015 at

Re: radosgw: bad content length

2015-01-30 Thread Yehuda Sadeh
Right, the we should call strict_strtoll() instead. I opened issue #10701, and sent a pull request for the fix. Thanks, Yehuda On Fri, Jan 30, 2015 at 10:01 AM, Axel Dunkel wrote: > > > >> > Telling from the source code the issue seems not to be that the > content >> > lengths do not match, rat

Re: radosgw: bad content length

2015-01-30 Thread Axel Dunkel
> > > Telling from the source code the issue seems not to be that the content > > lengths do not match, rather that the http-header Content-Length is not > > parseable (allthough the header is displayed fine in the debug output). If > > strict_strtol returns an error, to be exact - ? > > > >

Re: Supporting partial writes for EC backend

2015-01-30 Thread Bassam Tabbara
I’d like to explore adding a new EC pool type as you suggest. The idea is to have RBD sit on-top of a fully functional EC backend. Just as with RAID6 arrays today some applications are aware of the read-modify-write penalty and they could optimize for it. I’m new to ceph, so if you can point m

Re: dumpling integration : valgrind leak

2015-01-30 Thread Loic Dachary
After a little digging I found https://github.com/ceph/ceph/commit/02fae9fc54c10b5a932102bac43f32199d4cb612 which seems to be a plausible fix in dumpling also https://github.com/ceph/ceph/pull/3552. What do you think ? On 30/01/2015 16:35, Samuel Just wrote: > Yeah, that's probably new. > -Sam

AW: radosgw: bad content length

2015-01-30 Thread Axel Dunkel
Hi, the connection is not being shut down, also not idle (constantly transferring with more than 100 Mbit/s). It (really :-) ) is the special modified libapache2-mod-fastcgi, 2.4.7~0910052141-ceph1. Telling from the source code the issue seems not to be that the content lengths do not match, r

Re: radosgw: bad content length

2015-01-30 Thread Yehuda Sadeh
On Fri, Jan 30, 2015 at 8:58 AM, Axel Dunkel wrote: > Hi, > > the connection is not being shut down, also not idle (constantly > transferring with more than 100 Mbit/s). It (really :-) ) is the special > modified libapache2-mod-fastcgi, 2.4.7~0910052141-ceph1. > > Telling from the source code the

Re: Supporting partial writes for EC backend

2015-01-30 Thread Samuel Just
Basically, it's an architectural choice: https://github.com/ceph/ceph/blob/master/doc/dev/osd_internals/erasure_coding/pgbackend.rst#client-writes If we wanted to support partial writes, we'd probably want to introduce a second EC pool type with different tradeoffs. Note, you can get slow partia

Supporting partial writes for EC backend

2015-01-30 Thread Bassam Tabbara
Hello, I realize that the EC Backend does not support all operations like partial writes, truncate, and others. I’d like to understand whether adding the missing operations is something that is architecturally possible, or if this is a limitation of the architecture / approach. Can someone shed

Re: radosgw: bad content length

2015-01-30 Thread Yehuda Sadeh
On Fri, Jan 30, 2015 at 8:22 AM, Axel Dunkel wrote: > Hi, > > there are issues with radosgw and large file transfers without using > multiparts (like with "s3cmd --disable-multipart put") which seem to be > somehow known, but unsolved. > > Things run fine if the request takes no longer than 180sec

radosgw: bad content length

2015-01-30 Thread Axel Dunkel
Hi, there are issues with radosgw and large file transfers without using multiparts (like with "s3cmd --disable-multipart put") which seem to be somehow known, but unsolved. Things run fine if the request takes no longer than 180sec. If it takes longer, rgw_rest.cc (line 1236) gives the error

Re: [ceph-users] keyvaluestore backend metadata overhead

2015-01-30 Thread Haomai Wang
Although I still have some confusing, it's glad to see more attempts. More test results are welcomed! On Sat, Jan 31, 2015 at 12:08 AM, Chris Pacejo wrote: > On Fri, Jan 30, 2015 at 10:52 AM, Haomai Wang wrote: >> It's really a surprise that you impl a MySQL backend. Could I know the >> purpose

Re: [ceph-users] keyvaluestore backend metadata overhead

2015-01-30 Thread Chris Pacejo
On Fri, Jan 30, 2015 at 10:52 AM, Haomai Wang wrote: > It's really a surprise that you impl a MySQL backend. Could I know the > purpose? Because it may not fit with keyvaluestore I think. We've found it to perform better (in isolation) than LevelDB. We were able to map KeyValueDB's interface to

Re: [ceph-users] keyvaluestore backend metadata overhead

2015-01-30 Thread Haomai Wang
On Fri, Jan 30, 2015 at 11:41 PM, Chris Pacejo wrote: > Hi Haomai, > > On Thu, Jan 29, 2015 at 9:46 PM, Haomai Wang wrote: >> For metadata overhead, we need to resolve it at upper level, >> keyvaluestore won't add extra metadata in normal io except rarely >> header save which only update when hea

Re: [ceph-users] keyvaluestore backend metadata overhead

2015-01-30 Thread Chris Pacejo
Hi Haomai, On Thu, Jan 29, 2015 at 9:46 PM, Haomai Wang wrote: > For metadata overhead, we need to resolve it at upper level, > keyvaluestore won't add extra metadata in normal io except rarely > header save which only update when header changed. Unfortunately, our write workload is dominated by

Re: dumpling integration : valgrind leak

2015-01-30 Thread Samuel Just
Yeah, that's probably new. -Sam On Fri, Jan 30, 2015 at 7:00 AM, Loic Dachary wrote: > Hi Sam, > > I stumbled upon what seems to be a leak at > http://pulpito.ceph.com/loic-2015-01-29_15:41:06-rados-dumpling-backports---basic-multi/730305/ > and the valgrind xml file is at > http://tracker.cep

dumpling integration : valgrind leak

2015-01-30 Thread Loic Dachary
Hi Sam, I stumbled upon what seems to be a leak at http://pulpito.ceph.com/loic-2015-01-29_15:41:06-rados-dumpling-backports---basic-multi/730305/ and the valgrind xml file is at http://tracker.ceph.com/attachments/download/1610/v.txt 612 (584 direct, 28 indirect) bytes in 1 blocks are de

Re: [ceph-users] keyvaluestore backend metadata overhead

2015-01-30 Thread Chris Pacejo
Hi Sage, thanks for the quick reply. On Thu, Jan 29, 2015 at 5:51 PM, Sage Weil wrote: > There is a level (or two) of indirection in KeyValueStore's > GenericObjectMap that is there to allow object cloning. I wonder if we > will want to facilitate a backend that doesn't implement clone and can >

RE: mSHEC pull request (2nd challenge)

2015-01-30 Thread Miyamae, Takeshi
Hi Loic, > There was originally two files for tests and the one with 364 or something at > the end was removed > if I'm not mistaken. Is there a reason for that ? We had renamed the file from TestErasureCodeShec364.cc to TestErasureCodeShec_all.cc because the number of tests included in this fi

Re: mSHEC pull request (2nd challenge)

2015-01-30 Thread Loic Dachary
Hi, On 30/01/2015 11:28, Miyamae, Takeshi wrote: > Hi Loic, > > Thank you for your cooperation ! > If we could do something to reduce your burdens, please let us know. Hopefully tests come out clean, let see :-) There was originally two files for tests and the one with 364 or something at the

RE: mSHEC pull request (2nd challenge)

2015-01-30 Thread Miyamae, Takeshi
Hi Loic, Thank you for your cooperation ! If we could do something to reduce your burdens, please let us know. Best regards, Takeshi Miyamae -Original Message- From: Loic Dachary [mailto:l...@dachary.org] Sent: Thursday, January 29, 2015 11:38 PM To: Miyamae, Takeshi/宮前 剛 Cc: Ceph Devel