[ceph-users] radosgw get quota

2015-10-28 Thread Derek Yarnell 
I have had this issue before, and I don't think I have resolved it.  I
have been using the RGW admin api to set quota based on the docs[0].
But I can't seem to be able to get it to cough up and show me the quota
now.  Any ideas I get a 200 back but no body, I have tested this on a
Firefly (0.80.5-9) and Hammer (0.87.2-0) cluster.  The latter is what
the logs are for.

[0] - http://docs.ceph.com/docs/master/radosgw/adminops/#quotas

DEBUG:rgwadmin.rgw:URL:
http://ceph.umiacs.umd.edu/admin/user?quota&uid=derek"a-type=user
DEBUG:rgwadmin.rgw:Access Key: RTJ1TL13CH613JRU2PJD
DEBUG:rgwadmin.rgw:Verify: True  CA Bundle: None
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP
connection (1): ceph.umiacs.umd.edu
DEBUG:requests.packages.urllib3.connectionpool:"GET
/admin/user?quota&uid=derek"a-type=user HTTP/1.1" 200 0
INFO:rgwadmin.rgw:No JSON object could be decoded


2015-10-28 23:02:46.445367 7f444cff1700  1 civetweb: 0x7f445c026d00:
127.0.0.1 - - [28/Oct/2015:23:02:46 -0400] "GET /admin/user HTTP/1.1" -1
0 - python-requests/2.7.0 CPython/2.7.5 Linux/3.10.0-229.14.1.el7.x86_64
2015-10-28 23:03:02.063755 7f447ace2700  2
RGWDataChangesLog::ChangesRenewThread: start
2015-10-28 23:03:17.139339 7f443cfd1700 20 RGWEnv::set(): HTTP_HOST:
localhost:7480
2015-10-28 23:03:17.139357 7f443cfd1700 20 RGWEnv::set():
HTTP_ACCEPT_ENCODING: gzip, deflate
2015-10-28 23:03:17.139358 7f443cfd1700 20 RGWEnv::set(): HTTP_ACCEPT: */*
2015-10-28 23:03:17.139364 7f443cfd1700 20 RGWEnv::set():
HTTP_USER_AGENT: python-requests/2.7.0 CPython/2.7.5
Linux/3.10.0-229.14.1.el7.x86_64
2015-10-28 23:03:17.139375 7f443cfd1700 20 RGWEnv::set(): HTTP_DATE:
Thu, 29 Oct 2015 03:03:17 GMT
2015-10-28 23:03:17.139377 7f443cfd1700 20 RGWEnv::set():
HTTP_AUTHORIZATION: AWS RTJ1TL13CH613JRU2PJD:ZtDQkxc+Nqo04zVsNND0yx32lds=
2015-10-28 23:03:17.139381 7f443cfd1700 20 RGWEnv::set():
HTTP_X_FORWARDED_FOR: 128.8.132.4
2015-10-28 23:03:17.139383 7f443cfd1700 20 RGWEnv::set():
HTTP_X_FORWARDED_HOST: ceph.umiacs.umd.edu
2015-10-28 23:03:17.139385 7f443cfd1700 20 RGWEnv::set():
HTTP_X_FORWARDED_SERVER: cephproxy00.umiacs.umd.edu
2015-10-28 23:03:17.139387 7f443cfd1700 20 RGWEnv::set():
HTTP_CONNECTION: Keep-Alive
2015-10-28 23:03:17.139392 7f443cfd1700 20 RGWEnv::set():
REQUEST_METHOD: GET
2015-10-28 23:03:17.139394 7f443cfd1700 20 RGWEnv::set(): REQUEST_URI:
/admin/user
2015-10-28 23:03:17.139397 7f443cfd1700 20 RGWEnv::set(): QUERY_STRING:
quota&uid=derek"a-type=user
2015-10-28 23:03:17.139401 7f443cfd1700 20 RGWEnv::set(): REMOTE_USER:
2015-10-28 23:03:17.139403 7f443cfd1700 20 RGWEnv::set(): SCRIPT_URI:
/admin/user
2015-10-28 23:03:17.139408 7f443cfd1700 20 RGWEnv::set(): SERVER_PORT: 7480
2015-10-28 23:03:17.139409 7f443cfd1700 20 HTTP_ACCEPT=*/*
2015-10-28 23:03:17.139410 7f443cfd1700 20 HTTP_ACCEPT_ENCODING=gzip,
deflate
2015-10-28 23:03:17.139411 7f443cfd1700 20 HTTP_AUTHORIZATION=AWS
RTJ1TL13CH613JRU2PJD:ZtDQkxc+Nqo04zVsNND0yx32lds=
2015-10-28 23:03:17.139412 7f443cfd1700 20 HTTP_CONNECTION=Keep-Alive
2015-10-28 23:03:17.139412 7f443cfd1700 20 HTTP_DATE=Thu, 29 Oct 2015
03:03:17 GMT
2015-10-28 23:03:17.139413 7f443cfd1700 20 HTTP_HOST=localhost:7480
2015-10-28 23:03:17.139413 7f443cfd1700 20
HTTP_USER_AGENT=python-requests/2.7.0 CPython/2.7.5
Linux/3.10.0-229.14.1.el7.x86_64
2015-10-28 23:03:17.139414 7f443cfd1700 20 HTTP_X_FORWARDED_FOR=128.8.132.4
2015-10-28 23:03:17.139415 7f443cfd1700 20
HTTP_X_FORWARDED_HOST=ceph.umiacs.umd.edu
2015-10-28 23:03:17.139416 7f443cfd1700 20
HTTP_X_FORWARDED_SERVER=cephproxy00.umiacs.umd.edu
2015-10-28 23:03:17.139416 7f443cfd1700 20
QUERY_STRING=quota&uid=derek"a-type=user
2015-10-28 23:03:17.139417 7f443cfd1700 20 REMOTE_USER=
2015-10-28 23:03:17.139417 7f443cfd1700 20 REQUEST_METHOD=GET
2015-10-28 23:03:17.139418 7f443cfd1700 20 REQUEST_URI=/admin/user
2015-10-28 23:03:17.139419 7f443cfd1700 20 SCRIPT_URI=/admin/user
2015-10-28 23:03:17.139419 7f443cfd1700 20 SERVER_PORT=7480
2015-10-28 23:03:17.139422 7f443cfd1700 20 RGWEnv::set(): HTTP_HOST:
localhost:7480
2015-10-28 23:03:17.139423 7f443cfd1700 20 RGWEnv::set():
HTTP_ACCEPT_ENCODING: gzip, deflate
2015-10-28 23:03:17.139424 7f443cfd1700 20 RGWEnv::set(): HTTP_ACCEPT: */*
2015-10-28 23:03:17.139425 7f443cfd1700 20 RGWEnv::set():
HTTP_USER_AGENT: python-requests/2.7.0 CPython/2.7.5
Linux/3.10.0-229.14.1.el7.x86_64
2015-10-28 23:03:17.139426 7f443cfd1700 20 RGWEnv::set(): HTTP_DATE:
Thu, 29 Oct 2015 03:03:17 GMT
2015-10-28 23:03:17.139428 7f443cfd1700 20 RGWEnv::set():
HTTP_AUTHORIZATION: AWS RTJ1TL13CH613JRU2PJD:ZtDQkxc+Nqo04zVsNND0yx32lds=
2015-10-28 23:03:17.139429 7f443cfd1700 20 RGWEnv::set():
HTTP_X_FORWARDED_FOR: 128.8.132.4
2015-10-28 23:03:17.139430 7f443cfd1700 20 RGWEnv::set():
HTTP_X_FORWARDED_HOST: ceph.umiacs.umd.edu
2015-10-28 23:03:17.139431 7f443cfd1700 20 RGWEnv::set():
HTTP_X_FORWARDED_SERVER: cephproxy00.umiacs.umd.edu
2015-10-28 23:03:17.139432 7f443cfd1700 20 RGWEnv::set():
HTTP_CONNECTION: Keep-Alive
2015-10-28 23:03:17.139433 7f443c

Re: [ceph-users] radosgw get quota

2015-10-29 Thread Derek Yarnell 
Sorry, the information is in the headers.  So I think the valid question
to follow up is why is this information in the headers and not the body
of the request.  I think this is a bug, but maybe I am not aware of a
subtly.  It would seem this json comes from this line[0].

[0] -
https://github.com/ceph/ceph/blob/83e10f7e2df0a71bd59e6ef2aa06b52b186fddaa/src/rgw/rgw_rest_user.cc#L697

For example the information is returned in what seems to be the
Content-type header as follows.  Maybe the missing : in the json
encoding would explain something?

INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS
connection (1): ceph.umiacs.umd.edu
DEBUG:requests.packages.urllib3.connectionpool:"GET
/admin/user?quota&format=json&uid=foo1209"a-type=user HTTP/1.1" 200 0
INFO:rgwadmin.rgw:[('date', 'Thu, 29 Oct 2015 18:28:45 GMT'),
('{"enabled"', 'true,"max_size_kb":12345,"max_objects":-1}Content-type:
application/json'), ('content-length', '0'), ('server', 'Apache/2.4.6
(Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5')]

On 10/28/15 11:15 PM, Derek Yarnell wrote:
> I have had this issue before, and I don't think I have resolved it.  I
> have been using the RGW admin api to set quota based on the docs[0].
> But I can't seem to be able to get it to cough up and show me the quota
> now.  Any ideas I get a 200 back but no body, I have tested this on a
> Firefly (0.80.5-9) and Hammer (0.87.2-0) cluster.  The latter is what
> the logs are for.
> 
> [0] - http://docs.ceph.com/docs/master/radosgw/adminops/#quotas
> 
> DEBUG:rgwadmin.rgw:URL:
> http://ceph.umiacs.umd.edu/admin/user?quota&uid=derek"a-type=user
> DEBUG:rgwadmin.rgw:Access Key: RTJ1TL13CH613JRU2PJD
> DEBUG:rgwadmin.rgw:Verify: True  CA Bundle: None
> INFO:requests.packages.urllib3.connectionpool:Starting new HTTP
> connection (1): ceph.umiacs.umd.edu
> DEBUG:requests.packages.urllib3.connectionpool:"GET
> /admin/user?quota&uid=derek"a-type=user HTTP/1.1" 200 0
> INFO:rgwadmin.rgw:No JSON object could be decoded
> 
> 
> 2015-10-28 23:02:46.445367 7f444cff1700  1 civetweb: 0x7f445c026d00:
> 127.0.0.1 - - [28/Oct/2015:23:02:46 -0400] "GET /admin/user HTTP/1.1" -1
> 0 - python-requests/2.7.0 CPython/2.7.5 Linux/3.10.0-229.14.1.el7.x86_64
> 2015-10-28 23:03:02.063755 7f447ace2700  2
> RGWDataChangesLog::ChangesRenewThread: start
> 2015-10-28 23:03:17.139339 7f443cfd1700 20 RGWEnv::set(): HTTP_HOST:
> localhost:7480
> 2015-10-28 23:03:17.139357 7f443cfd1700 20 RGWEnv::set():
> HTTP_ACCEPT_ENCODING: gzip, deflate
> 2015-10-28 23:03:17.139358 7f443cfd1700 20 RGWEnv::set(): HTTP_ACCEPT: */*
> 2015-10-28 23:03:17.139364 7f443cfd1700 20 RGWEnv::set():
> HTTP_USER_AGENT: python-requests/2.7.0 CPython/2.7.5
> Linux/3.10.0-229.14.1.el7.x86_64
> 2015-10-28 23:03:17.139375 7f443cfd1700 20 RGWEnv::set(): HTTP_DATE:
> Thu, 29 Oct 2015 03:03:17 GMT
> 2015-10-28 23:03:17.139377 7f443cfd1700 20 RGWEnv::set():
> HTTP_AUTHORIZATION: AWS RTJ1TL13CH613JRU2PJD:ZtDQkxc+Nqo04zVsNND0yx32lds=
> 2015-10-28 23:03:17.139381 7f443cfd1700 20 RGWEnv::set():
> HTTP_X_FORWARDED_FOR: 128.8.132.4
> 2015-10-28 23:03:17.139383 7f443cfd1700 20 RGWEnv::set():
> HTTP_X_FORWARDED_HOST: ceph.umiacs.umd.edu
> 2015-10-28 23:03:17.139385 7f443cfd1700 20 RGWEnv::set():
> HTTP_X_FORWARDED_SERVER: cephproxy00.umiacs.umd.edu
> 2015-10-28 23:03:17.139387 7f443cfd1700 20 RGWEnv::set():
> HTTP_CONNECTION: Keep-Alive
> 2015-10-28 23:03:17.139392 7f443cfd1700 20 RGWEnv::set():
> REQUEST_METHOD: GET
> 2015-10-28 23:03:17.139394 7f443cfd1700 20 RGWEnv::set(): REQUEST_URI:
> /admin/user
> 2015-10-28 23:03:17.139397 7f443cfd1700 20 RGWEnv::set(): QUERY_STRING:
> quota&uid=derek"a-type=user
> 2015-10-28 23:03:17.139401 7f443cfd1700 20 RGWEnv::set(): REMOTE_USER:
> 2015-10-28 23:03:17.139403 7f443cfd1700 20 RGWEnv::set(): SCRIPT_URI:
> /admin/user
> 2015-10-28 23:03:17.139408 7f443cfd1700 20 RGWEnv::set(): SERVER_PORT: 7480
> 2015-10-28 23:03:17.139409 7f443cfd1700 20 HTTP_ACCEPT=*/*
> 2015-10-28 23:03:17.139410 7f443cfd1700 20 HTTP_ACCEPT_ENCODING=gzip,
> deflate
> 2015-10-28 23:03:17.139411 7f443cfd1700 20 HTTP_AUTHORIZATION=AWS
> RTJ1TL13CH613JRU2PJD:ZtDQkxc+Nqo04zVsNND0yx32lds=
> 2015-10-28 23:03:17.139412 7f443cfd1700 20 HTTP_CONNECTION=Keep-Alive
> 2015-10-28 23:03:17.139412 7f443cfd1700 20 HTTP_DATE=Thu, 29 Oct 2015
> 03:03:17 GMT
> 2015-10-28 23:03:17.139413 7f443cfd1700 20 HTTP_HOST=localhost:7480
> 2015-10-28 23:03:17.139413 7f443cfd1700 20
> HTTP_USER_AGENT=python-requests/2.7.0 CPython/2.7.5
> Linux/3.10.0-229.14.1.el7.x86_64
> 2015-10-28 23:03:17.139414 7f443cfd1700 20 HTTP_X_FORWARDED_FOR=128.8.132.4
> 2015-10-28 23:03:17.139415 7f443cfd1700 20
> HTTP_X_FORWARDED_HOST=ceph.umiacs.umd.edu
> 2015-10-28 23:03:17.139416 7f443cfd1700 20
> HTTP_X_FORWARDED_SERVER=cephproxy00.umiacs.umd.edu
> 2015-10-28 23:03:17.139416 7f443cfd1700 20
> QUERY_STRING=quota&uid=derek"a-type=user
> 2015-10-28 23:03:17.139417 7f443cfd1700 20 REMOTE_USER=
> 2015-10-28 23:03:17.139417 7f443cfd1700 20 REQUEST_METHO

Re: [ceph-users] radosgw get quota

2015-10-29 Thread Yehuda Sadeh-Weinraub 
On Thu, Oct 29, 2015 at 11:29 AM, Derek Yarnell  wrote:
> Sorry, the information is in the headers.  So I think the valid question
> to follow up is why is this information in the headers and not the body
> of the request.  I think this is a bug, but maybe I am not aware of a
> subtly.  It would seem this json comes from this line[0].
>
> [0] -
> https://github.com/ceph/ceph/blob/83e10f7e2df0a71bd59e6ef2aa06b52b186fddaa/src/rgw/rgw_rest_user.cc#L697
>
> For example the information is returned in what seems to be the
> Content-type header as follows.  Maybe the missing : in the json
> encoding would explain something?

It's definitely a bug. It looks like we fail to call end_header()
before it, so everything is dumped before we close the http header.
Can you open a ceph tracker issue with the info you provided here?

Thanks,
Yehuda

>
> INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS
> connection (1): ceph.umiacs.umd.edu
> DEBUG:requests.packages.urllib3.connectionpool:"GET
> /admin/user?quota&format=json&uid=foo1209"a-type=user HTTP/1.1" 200 0
> INFO:rgwadmin.rgw:[('date', 'Thu, 29 Oct 2015 18:28:45 GMT'),
> ('{"enabled"', 'true,"max_size_kb":12345,"max_objects":-1}Content-type:
> application/json'), ('content-length', '0'), ('server', 'Apache/2.4.6
> (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5')]
>
> On 10/28/15 11:15 PM, Derek Yarnell wrote:
>> I have had this issue before, and I don't think I have resolved it.  I
>> have been using the RGW admin api to set quota based on the docs[0].
>> But I can't seem to be able to get it to cough up and show me the quota
>> now.  Any ideas I get a 200 back but no body, I have tested this on a
>> Firefly (0.80.5-9) and Hammer (0.87.2-0) cluster.  The latter is what
>> the logs are for.
>>
>> [0] - http://docs.ceph.com/docs/master/radosgw/adminops/#quotas
>>
>> DEBUG:rgwadmin.rgw:URL:
>> http://ceph.umiacs.umd.edu/admin/user?quota&uid=derek"a-type=user
>> DEBUG:rgwadmin.rgw:Access Key: RTJ1TL13CH613JRU2PJD
>> DEBUG:rgwadmin.rgw:Verify: True  CA Bundle: None
>> INFO:requests.packages.urllib3.connectionpool:Starting new HTTP
>> connection (1): ceph.umiacs.umd.edu
>> DEBUG:requests.packages.urllib3.connectionpool:"GET
>> /admin/user?quota&uid=derek"a-type=user HTTP/1.1" 200 0
>> INFO:rgwadmin.rgw:No JSON object could be decoded
>>
>>
>> 2015-10-28 23:02:46.445367 7f444cff1700  1 civetweb: 0x7f445c026d00:
>> 127.0.0.1 - - [28/Oct/2015:23:02:46 -0400] "GET /admin/user HTTP/1.1" -1
>> 0 - python-requests/2.7.0 CPython/2.7.5 Linux/3.10.0-229.14.1.el7.x86_64
>> 2015-10-28 23:03:02.063755 7f447ace2700  2
>> RGWDataChangesLog::ChangesRenewThread: start
>> 2015-10-28 23:03:17.139339 7f443cfd1700 20 RGWEnv::set(): HTTP_HOST:
>> localhost:7480
>> 2015-10-28 23:03:17.139357 7f443cfd1700 20 RGWEnv::set():
>> HTTP_ACCEPT_ENCODING: gzip, deflate
>> 2015-10-28 23:03:17.139358 7f443cfd1700 20 RGWEnv::set(): HTTP_ACCEPT: */*
>> 2015-10-28 23:03:17.139364 7f443cfd1700 20 RGWEnv::set():
>> HTTP_USER_AGENT: python-requests/2.7.0 CPython/2.7.5
>> Linux/3.10.0-229.14.1.el7.x86_64
>> 2015-10-28 23:03:17.139375 7f443cfd1700 20 RGWEnv::set(): HTTP_DATE:
>> Thu, 29 Oct 2015 03:03:17 GMT
>> 2015-10-28 23:03:17.139377 7f443cfd1700 20 RGWEnv::set():
>> HTTP_AUTHORIZATION: AWS RTJ1TL13CH613JRU2PJD:ZtDQkxc+Nqo04zVsNND0yx32lds=
>> 2015-10-28 23:03:17.139381 7f443cfd1700 20 RGWEnv::set():
>> HTTP_X_FORWARDED_FOR: 128.8.132.4
>> 2015-10-28 23:03:17.139383 7f443cfd1700 20 RGWEnv::set():
>> HTTP_X_FORWARDED_HOST: ceph.umiacs.umd.edu
>> 2015-10-28 23:03:17.139385 7f443cfd1700 20 RGWEnv::set():
>> HTTP_X_FORWARDED_SERVER: cephproxy00.umiacs.umd.edu
>> 2015-10-28 23:03:17.139387 7f443cfd1700 20 RGWEnv::set():
>> HTTP_CONNECTION: Keep-Alive
>> 2015-10-28 23:03:17.139392 7f443cfd1700 20 RGWEnv::set():
>> REQUEST_METHOD: GET
>> 2015-10-28 23:03:17.139394 7f443cfd1700 20 RGWEnv::set(): REQUEST_URI:
>> /admin/user
>> 2015-10-28 23:03:17.139397 7f443cfd1700 20 RGWEnv::set(): QUERY_STRING:
>> quota&uid=derek"a-type=user
>> 2015-10-28 23:03:17.139401 7f443cfd1700 20 RGWEnv::set(): REMOTE_USER:
>> 2015-10-28 23:03:17.139403 7f443cfd1700 20 RGWEnv::set(): SCRIPT_URI:
>> /admin/user
>> 2015-10-28 23:03:17.139408 7f443cfd1700 20 RGWEnv::set(): SERVER_PORT: 7480
>> 2015-10-28 23:03:17.139409 7f443cfd1700 20 HTTP_ACCEPT=*/*
>> 2015-10-28 23:03:17.139410 7f443cfd1700 20 HTTP_ACCEPT_ENCODING=gzip,
>> deflate
>> 2015-10-28 23:03:17.139411 7f443cfd1700 20 HTTP_AUTHORIZATION=AWS
>> RTJ1TL13CH613JRU2PJD:ZtDQkxc+Nqo04zVsNND0yx32lds=
>> 2015-10-28 23:03:17.139412 7f443cfd1700 20 HTTP_CONNECTION=Keep-Alive
>> 2015-10-28 23:03:17.139412 7f443cfd1700 20 HTTP_DATE=Thu, 29 Oct 2015
>> 03:03:17 GMT
>> 2015-10-28 23:03:17.139413 7f443cfd1700 20 HTTP_HOST=localhost:7480
>> 2015-10-28 23:03:17.139413 7f443cfd1700 20
>> HTTP_USER_AGENT=python-requests/2.7.0 CPython/2.7.5
>> Linux/3.10.0-229.14.1.el7.x86_64
>> 2015-10-28 23:03:17.139414 7f443cfd1700 20 HTTP_X_FORWARDED_FOR=128.8.132.4
>> 2015-10-2