[Yahoo-eng-team] [Bug 1202785] Re: Authentication is not checked before sending potentially large request bodies
** Changed in: glance Status: Triaged => Confirmed ** Changed in: glance Status: Confirmed => Won't Fix ** Changed in: glance Importance: High => Undecided -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Glance. https://bugs.launchpad.net/bugs/1202785 Title: Authentication is not checked before sending potentially large request bodies Status in OpenStack Image Registry and Delivery Service (Glance): Won't Fix Status in OpenStack Identity (Keystone) Middleware: Incomplete Status in OpenStack Security Advisories: Won't Fix Status in Python client library for Keystone: Invalid Bug description: When making an HTTP request with a body to an api using the keystone auth_token middleware and no request size limiting then an unauthorized user can send a very large request that will not fail with a 401 until after all of the data is sent. This means that anyone who can hit an api could make many requests with large bodies and not be denied until after all of that data has been sent, wasting lots/all of the resources on the api node essentially bringing it down. This issue can be mitigated for apis like nova by having middleware or using the webserver to limit the maximum size of a request. In the case of the glance-api however, large requests such as image uploads need to occur. Perhaps the auth_token middleware should look at request headers and perform authN and authZ before accepting all of the request body. It's also very inefficient and time consuming to wait until all the data is sent before receiving a 401. I am not sure of the level of impact this could have for most deployers and the different APIs. Here is an example of requests to glance and devstack with a bad token and their times to complete. Nova-api on devstack also accepted large bodies before returning a 401. 1 Meg Image [ameade@ameade-dev:~] [17:30:16] $ time glance --debug --os-auth-token 'gah' image-create --name test <1meg.img curl -i -X POST -H 'Transfer-Encoding: chunked' -H 'User-Agent: python-glanceclient' -H 'x-image-meta-size: 1048576' -H 'x-image-meta-is_public: False' -H 'X-Auth-Token: gah' -H 'Content-Type: application/octet-stream' -H 'x-image-meta-name: test' -d '', mode 'r' at 0x7f8d762bd150>' http://50.56.173.46:9292/v1/images HTTP/1.1 401 Unauthorized date: Thu, 18 Jul 2013 17:30:30 GMT content-length: 253 content-type: text/plain; charset=UTF-8 401 Unauthorized This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. Request returned failure status. Invalid OpenStack Identity credentials. real0m0.766s user0m0.312s sys 0m0.164s 100 meg [ameade@ameade-dev:~] [17:31:35] $ time glance --debug --os-auth-token 'gah' image-create --name test <100meg.img curl -i -X POST -H 'Transfer-Encoding: chunked' -H 'User-Agent: python-glanceclient' -H 'x-image-meta-size: 104857600' -H 'x-image-meta-is_public: False' -H 'X-Auth-Token: gah' -H 'Content-Type: application/octet-stream' -H 'x-image-meta-name: test' -d '', mode 'r' at 0x7f6af9768150>' http://50.56.173.46:9292/v1/images HTTP/1.1 401 Unauthorized date: Thu, 18 Jul 2013 17:31:40 GMT content-length: 253 content-type: text/plain; charset=UTF-8 401 Unauthorized This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. Request returned failure status. Invalid OpenStack Identity credentials. real0m1.441s user0m0.420s sys 0m0.344s 10 gig [ameade@ameade-dev:~] [17:16:23] 1 $ time glance --debug --os-auth-token 'gah' image-create --name test <10g.img curl -i -X POST -H 'Transfer-Encoding: chunked' -H 'User-Agent: python-glanceclient' -H 'x-image-meta-size: 100' -H 'x-image-meta-is_public: False' -H 'X-Auth-Token: gah' -H 'Content-Type: application/octet-stream' -H 'x-image-meta-name: test' -d '', mode 'r' at 0x7f768c151150>' http://50.56.173.46:9292/v1/images HTTP/1.1 401 Unauthorized date: Thu, 18 Jul 2013 17:16:28 GMT content-length: 253 content-type: text/plain; charset=UTF-8 401 Unauthorized This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. Request returned failure status. Invalid OpenStack Identity credentials. real0m56.082s user0m6.308s sys 0m17.669s To manage notifications about this bug go to: https://bugs.launchpad.net/glance/+bug/120278
[Yahoo-eng-team] [Bug 1202785] Re: Authentication is not checked before sending potentially large request bodies
** Changed in: keystonemiddleware Status: Incomplete => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Glance. https://bugs.launchpad.net/bugs/1202785 Title: Authentication is not checked before sending potentially large request bodies Status in OpenStack Image Registry and Delivery Service (Glance): Won't Fix Status in OpenStack Identity (Keystone) Middleware: Invalid Status in OpenStack Security Advisories: Won't Fix Status in Python client library for Keystone: Invalid Bug description: When making an HTTP request with a body to an api using the keystone auth_token middleware and no request size limiting then an unauthorized user can send a very large request that will not fail with a 401 until after all of the data is sent. This means that anyone who can hit an api could make many requests with large bodies and not be denied until after all of that data has been sent, wasting lots/all of the resources on the api node essentially bringing it down. This issue can be mitigated for apis like nova by having middleware or using the webserver to limit the maximum size of a request. In the case of the glance-api however, large requests such as image uploads need to occur. Perhaps the auth_token middleware should look at request headers and perform authN and authZ before accepting all of the request body. It's also very inefficient and time consuming to wait until all the data is sent before receiving a 401. I am not sure of the level of impact this could have for most deployers and the different APIs. Here is an example of requests to glance and devstack with a bad token and their times to complete. Nova-api on devstack also accepted large bodies before returning a 401. 1 Meg Image [ameade@ameade-dev:~] [17:30:16] $ time glance --debug --os-auth-token 'gah' image-create --name test <1meg.img curl -i -X POST -H 'Transfer-Encoding: chunked' -H 'User-Agent: python-glanceclient' -H 'x-image-meta-size: 1048576' -H 'x-image-meta-is_public: False' -H 'X-Auth-Token: gah' -H 'Content-Type: application/octet-stream' -H 'x-image-meta-name: test' -d '', mode 'r' at 0x7f8d762bd150>' http://50.56.173.46:9292/v1/images HTTP/1.1 401 Unauthorized date: Thu, 18 Jul 2013 17:30:30 GMT content-length: 253 content-type: text/plain; charset=UTF-8 401 Unauthorized This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. Request returned failure status. Invalid OpenStack Identity credentials. real0m0.766s user0m0.312s sys 0m0.164s 100 meg [ameade@ameade-dev:~] [17:31:35] $ time glance --debug --os-auth-token 'gah' image-create --name test <100meg.img curl -i -X POST -H 'Transfer-Encoding: chunked' -H 'User-Agent: python-glanceclient' -H 'x-image-meta-size: 104857600' -H 'x-image-meta-is_public: False' -H 'X-Auth-Token: gah' -H 'Content-Type: application/octet-stream' -H 'x-image-meta-name: test' -d '', mode 'r' at 0x7f6af9768150>' http://50.56.173.46:9292/v1/images HTTP/1.1 401 Unauthorized date: Thu, 18 Jul 2013 17:31:40 GMT content-length: 253 content-type: text/plain; charset=UTF-8 401 Unauthorized This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. Request returned failure status. Invalid OpenStack Identity credentials. real0m1.441s user0m0.420s sys 0m0.344s 10 gig [ameade@ameade-dev:~] [17:16:23] 1 $ time glance --debug --os-auth-token 'gah' image-create --name test <10g.img curl -i -X POST -H 'Transfer-Encoding: chunked' -H 'User-Agent: python-glanceclient' -H 'x-image-meta-size: 100' -H 'x-image-meta-is_public: False' -H 'X-Auth-Token: gah' -H 'Content-Type: application/octet-stream' -H 'x-image-meta-name: test' -d '', mode 'r' at 0x7f768c151150>' http://50.56.173.46:9292/v1/images HTTP/1.1 401 Unauthorized date: Thu, 18 Jul 2013 17:16:28 GMT content-length: 253 content-type: text/plain; charset=UTF-8 401 Unauthorized This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. Request returned failure status. Invalid OpenStack Identity credentials. real0m56.082s user0m6.308s sys 0m17.669s To manage notifications about this bug go to: https://bugs.launchpad.net/glance/+bug/1202785/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@list