[openstack-dev] Installation Error : keystone_role provider 'openstack': Could not authenticate
Hi, I'm installing OpenStack using packstack, and during the installation I'm facing an issue, keystone is not installing to completion. It gives the following error: ERROR : Error appeared during Puppet run: 10.16.6.33_keystone.pp > Error: Could not prefetch keystone_role provider 'openstack': Could not > authenticate It also says that the file /root/keystonerc_admin has been created, but there is no such file present in that location. Could someone guide me on this, as I did not find this issue discussed on any forum. Thanks, Dhvanan Shah __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Keystone Authorization Failed: Forbidden (HTTP 403)
Hi, Problem resolved. curl 10.16.37.221:5000 returned access denied. So I added a no proxy for the host ip in the browser after which it began returning the JSON data. After this I exported the same in my keystone_adminrc file as Chinmaya pointed out. That solved the problem and it no longer gives the forbidden error. But the funny thing here is that I have had this setup running for quite some time now and I have not added a no_proxy for the host ip and I also haven't faced this issue before. So I am not sure what triggered this error here now. Thanks a lot for your inputs. On Wed, Apr 27, 2016 at 5:30 PM, Dolph Mathews wrote: > > On Wed, Apr 27, 2016 at 6:53 AM, Dhvanan Shah wrote: > >> Hi, >> >> Enabling the debug flag didn't give any additional information. >> >> 2 node Cluster means that I have one controller that also runs the >> compute and an additional compute node, thus 2 node OpenStack Cluster. >> >> The problem here is not with the password as I am able to log in through >> the dashboard. Any action performed gives a Forbidden error and >> authorization failed for keystone. >> >> Any other things that I could look at? >> > > Another long shot, but you might have an unintended surprise in your > environment. > > $ env | grep ^OS_ > > More likely though, I'm guessing you don't actually have the "admin" role > on the "admin" tenant that you're expecting. The 403 is indicating that you > are authenticated successfully (your password is correct), but you don't > have authorization to make the request (listing users, for example). You'd > be able to login to horizon and spin up a VM, or do the same from the CLI, > but not make the requests you're using to exercise the cloud admin role. > > >> On Wed, Apr 27, 2016 at 4:55 PM, Dolph Mathews >> wrote: >> >>> Depending on which release of keystone you're running, try enabling >>> either insecure_debug (more recent releases) or debug (older releases) to >>> true in keystone.conf to get more detailed error messages from keystone. >>> >>> >>> https://github.com/openstack/keystone/blob/3c4fe622ac5da00b04ccc8bc4e207a2e9ab0f863/etc/keystone.conf.sample#L87-L91 >>> >>> That said, your configuration looks entirely correct to me, so I'm >>> curious what the outcome is here. The only other red flag I see is that you >>> mentioned a "2 node OpenStack cluster", and I'm not sure what that means in >>> this context, exactly. How are the 2 nodes utilized? >>> >>> On Wed, Apr 27, 2016 at 5:43 AM, Dhvanan Shah wrote: >>> >>>> keystone --debug user-list gives this: >>>> >>>> /usr/lib/python2.7/site-packages/keystoneclient/shell.py:65: >>>> DeprecationWarning: The keystone CLI is deprecated in favor of >>>> python-openstackclient. For a Python library, continue using >>>> python-keystoneclient. >>>> 'python-keystoneclient.', DeprecationWarning) >>>> DEBUG:keystoneclient.auth.identity.v2:Making authentication request to >>>> http://10.16.37.221:5000/v2.0/tokens >>>> INFO:requests.packages.urllib3.connectionpool:Starting new HTTP >>>> connection (1): proxy.serc.iisc.ernet.in >>>> DEBUG:requests.packages.urllib3.connectionpool:"POST >>>> http://10.16.37.221:5000/v2.0/tokens HTTP/1.1" 403 3370 >>>> DEBUG:keystoneclient.session:Request returned failure status: 403 >>>> Authorization Failed: Forbidden (HTTP 403) >>>> >>>> nova --debug user list gives this: >>>> >>>> DEBUG (session:195) REQ: curl -g -i -X GET >>>> http://10.16.37.221:5000/v2.0 -H "Accept: application/json" -H >>>> "User-Agent: python-keystoneclient" >>>> INFO (connectionpool:203) Starting new HTTP connection (1): >>>> proxy.serc.iisc.ernet.in >>>> DEBUG (connectionpool:383) "GET http://10.16.37.221:5000/v2.0 >>>> HTTP/1.1" 403 3275 >>>> DEBUG (session:224) RESP: >>>> DEBUG (session:396) Request returned failure status: 403 >>>> WARNING (base:133) Discovering versions from the identity service >>>> failed when creating the password plugin. Attempting to determine version >>>> from URL. >>>> DEBUG (v2:76) Making authentication request to >>>> http://10.16.37.221:5000/v2.0/tokens >>>> DEBUG (connectionpool:383) "POST http://10.16.37.221:5000/v2.0/tokens >>>> HTTP/1.1" 403 3370 >>>&g
Re: [openstack-dev] Keystone Authorization Failed: Forbidden (HTTP 403)
Hi, Enabling the debug flag didn't give any additional information. 2 node Cluster means that I have one controller that also runs the compute and an additional compute node, thus 2 node OpenStack Cluster. The problem here is not with the password as I am able to log in through the dashboard. Any action performed gives a Forbidden error and authorization failed for keystone. Any other things that I could look at? On Wed, Apr 27, 2016 at 4:55 PM, Dolph Mathews wrote: > Depending on which release of keystone you're running, try enabling either > insecure_debug (more recent releases) or debug (older releases) to true in > keystone.conf to get more detailed error messages from keystone. > > > https://github.com/openstack/keystone/blob/3c4fe622ac5da00b04ccc8bc4e207a2e9ab0f863/etc/keystone.conf.sample#L87-L91 > > That said, your configuration looks entirely correct to me, so I'm curious > what the outcome is here. The only other red flag I see is that you > mentioned a "2 node OpenStack cluster", and I'm not sure what that means in > this context, exactly. How are the 2 nodes utilized? > > On Wed, Apr 27, 2016 at 5:43 AM, Dhvanan Shah wrote: > >> keystone --debug user-list gives this: >> >> /usr/lib/python2.7/site-packages/keystoneclient/shell.py:65: >> DeprecationWarning: The keystone CLI is deprecated in favor of >> python-openstackclient. For a Python library, continue using >> python-keystoneclient. >> 'python-keystoneclient.', DeprecationWarning) >> DEBUG:keystoneclient.auth.identity.v2:Making authentication request to >> http://10.16.37.221:5000/v2.0/tokens >> INFO:requests.packages.urllib3.connectionpool:Starting new HTTP >> connection (1): proxy.serc.iisc.ernet.in >> DEBUG:requests.packages.urllib3.connectionpool:"POST >> http://10.16.37.221:5000/v2.0/tokens HTTP/1.1" 403 3370 >> DEBUG:keystoneclient.session:Request returned failure status: 403 >> Authorization Failed: Forbidden (HTTP 403) >> >> nova --debug user list gives this: >> >> DEBUG (session:195) REQ: curl -g -i -X GET http://10.16.37.221:5000/v2.0 >> -H "Accept: application/json" -H "User-Agent: python-keystoneclient" >> INFO (connectionpool:203) Starting new HTTP connection (1): >> proxy.serc.iisc.ernet.in >> DEBUG (connectionpool:383) "GET http://10.16.37.221:5000/v2.0 HTTP/1.1" >> 403 3275 >> DEBUG (session:224) RESP: >> DEBUG (session:396) Request returned failure status: 403 >> WARNING (base:133) Discovering versions from the identity service failed >> when creating the password plugin. Attempting to determine version from URL. >> DEBUG (v2:76) Making authentication request to >> http://10.16.37.221:5000/v2.0/tokens >> DEBUG (connectionpool:383) "POST http://10.16.37.221:5000/v2.0/tokens >> HTTP/1.1" 403 3370 >> DEBUG (session:396) Request returned failure status: 403 >> DEBUG (shell:914) Forbidden (HTTP 403) >> Forbidden: Forbidden (HTTP 403) >> ERROR (Forbidden): Forbidden (HTTP 403) >> >> >> >> On Wed, Apr 27, 2016 at 3:12 PM, Dhvanan Shah wrote: >> >>> On running openstack-status this is what I get (all the services are >>> running, so not included that here) >>> >>> == Keystone users == >>> /usr/lib/python2.7/site-packages/keystoneclient/shell.py:65: >>> DeprecationWarning: The keystone CLI is deprecated in favor of >>> python-openstackclient. For a Python library, continue using >>> python-keystoneclient. >>> 'python-keystoneclient.', DeprecationWarning) >>> Authorization Failed: Forbidden (HTTP 403) >>> == Glance images == >>> Forbidden (HTTP 403) >>> == Nova managed services == >>> No handlers could be found for logger >>> "keystoneclient.auth.identity.generic.base" >>> ERROR (Forbidden): Forbidden (HTTP 403) >>> == Nova networks == >>> No handlers could be found for logger >>> "keystoneclient.auth.identity.generic.base" >>> ERROR (Forbidden): Forbidden (HTTP 403) >>> == Nova instance flavors == >>> No handlers could be found for logger >>> "keystoneclient.auth.identity.generic.base" >>> ERROR (Forbidden): Forbidden (HTTP 403) >>> == Nova instances == >>> No handlers could be found for logger >>> "keystoneclient.auth.identity.generic.base" >>> ERROR (Forbidden): Forbidden (HTTP 403) >>> >>> >>> On Wed, Apr 27, 2016 at 3:09 PM, Dhvanan Shah wrote: >>> >>>> Hi Jens, >>>> >>>> The
Re: [openstack-dev] Keystone Authorization Failed: Forbidden (HTTP 403)
keystone --debug user-list gives this: /usr/lib/python2.7/site-packages/keystoneclient/shell.py:65: DeprecationWarning: The keystone CLI is deprecated in favor of python-openstackclient. For a Python library, continue using python-keystoneclient. 'python-keystoneclient.', DeprecationWarning) DEBUG:keystoneclient.auth.identity.v2:Making authentication request to http://10.16.37.221:5000/v2.0/tokens INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): proxy.serc.iisc.ernet.in DEBUG:requests.packages.urllib3.connectionpool:"POST http://10.16.37.221:5000/v2.0/tokens HTTP/1.1" 403 3370 DEBUG:keystoneclient.session:Request returned failure status: 403 Authorization Failed: Forbidden (HTTP 403) nova --debug user list gives this: DEBUG (session:195) REQ: curl -g -i -X GET http://10.16.37.221:5000/v2.0 -H "Accept: application/json" -H "User-Agent: python-keystoneclient" INFO (connectionpool:203) Starting new HTTP connection (1): proxy.serc.iisc.ernet.in DEBUG (connectionpool:383) "GET http://10.16.37.221:5000/v2.0 HTTP/1.1" 403 3275 DEBUG (session:224) RESP: DEBUG (session:396) Request returned failure status: 403 WARNING (base:133) Discovering versions from the identity service failed when creating the password plugin. Attempting to determine version from URL. DEBUG (v2:76) Making authentication request to http://10.16.37.221:5000/v2.0/tokens DEBUG (connectionpool:383) "POST http://10.16.37.221:5000/v2.0/tokens HTTP/1.1" 403 3370 DEBUG (session:396) Request returned failure status: 403 DEBUG (shell:914) Forbidden (HTTP 403) Forbidden: Forbidden (HTTP 403) ERROR (Forbidden): Forbidden (HTTP 403) On Wed, Apr 27, 2016 at 3:12 PM, Dhvanan Shah wrote: > On running openstack-status this is what I get (all the services are > running, so not included that here) > > == Keystone users == > /usr/lib/python2.7/site-packages/keystoneclient/shell.py:65: > DeprecationWarning: The keystone CLI is deprecated in favor of > python-openstackclient. For a Python library, continue using > python-keystoneclient. > 'python-keystoneclient.', DeprecationWarning) > Authorization Failed: Forbidden (HTTP 403) > == Glance images == > Forbidden (HTTP 403) > == Nova managed services == > No handlers could be found for logger > "keystoneclient.auth.identity.generic.base" > ERROR (Forbidden): Forbidden (HTTP 403) > == Nova networks == > No handlers could be found for logger > "keystoneclient.auth.identity.generic.base" > ERROR (Forbidden): Forbidden (HTTP 403) > == Nova instance flavors == > No handlers could be found for logger > "keystoneclient.auth.identity.generic.base" > ERROR (Forbidden): Forbidden (HTTP 403) > == Nova instances == > No handlers could be found for logger > "keystoneclient.auth.identity.generic.base" > ERROR (Forbidden): Forbidden (HTTP 403) > > > On Wed, Apr 27, 2016 at 3:09 PM, Dhvanan Shah wrote: > >> Hi Jens, >> >> The password is correct when I echo $OS_PASSWORD. >> I downloaded the admin-openrc.sh file from the dashboard and sourced. I >> ran a nova list after that: >> No handlers could be found for logger >> "keystoneclient.auth.identity.generic.base" >> ERROR (Forbidden): Forbidden (HTTP 403) >> >> It still gives the error of forbidden access. >> I think the password is not the issue. Forbidden access might be >> something else. Do you want me to share anything else? >> >> On Wed, Apr 27, 2016 at 2:56 PM, Jens Rosenboom >> wrote: >> >>> 2016-04-27 10:30 GMT+02:00 Dhvanan Shah : >>> > UPDATE: >>> > I am able to log into Horizon and perform all actions without any >>> issue but >>> > on my terminal, I am not able to do the same. The password that I >>> thought >>> > was wrong is not the issue as I logged in with the same password. >>> > My keystone_adminrc file looks like this: >>> > >>> > unset OS_SERVICE_TOKEN OS_SERVICE_ENDPOINT >>> > export OS_USERNAME=admin >>> > export OS_PASSWORD= >>> > export OS_AUTH_URL=http://10.16.37.221:35357/v2.0 >>> > export PS1='[\u@\h \W(keystone_admin)]\$ ' >>> > >>> > export OS_TENANT_NAME=admin >>> > export OS_REGION_NAME=RegionOne >>> > >>> > >>> > Please suggest what I could do! >>> >>> Does your password contain special characters that might get mangled >>> by the shell? You could compare the output of "echo $OS_PASSWORD" to >>> verify. >>> >>> Otherwise, if the dashboard is working for you, you can go to >>>
Re: [openstack-dev] Keystone Authorization Failed: Forbidden (HTTP 403)
On running openstack-status this is what I get (all the services are running, so not included that here) == Keystone users == /usr/lib/python2.7/site-packages/keystoneclient/shell.py:65: DeprecationWarning: The keystone CLI is deprecated in favor of python-openstackclient. For a Python library, continue using python-keystoneclient. 'python-keystoneclient.', DeprecationWarning) Authorization Failed: Forbidden (HTTP 403) == Glance images == Forbidden (HTTP 403) == Nova managed services == No handlers could be found for logger "keystoneclient.auth.identity.generic.base" ERROR (Forbidden): Forbidden (HTTP 403) == Nova networks == No handlers could be found for logger "keystoneclient.auth.identity.generic.base" ERROR (Forbidden): Forbidden (HTTP 403) == Nova instance flavors == No handlers could be found for logger "keystoneclient.auth.identity.generic.base" ERROR (Forbidden): Forbidden (HTTP 403) == Nova instances == No handlers could be found for logger "keystoneclient.auth.identity.generic.base" ERROR (Forbidden): Forbidden (HTTP 403) On Wed, Apr 27, 2016 at 3:09 PM, Dhvanan Shah wrote: > Hi Jens, > > The password is correct when I echo $OS_PASSWORD. > I downloaded the admin-openrc.sh file from the dashboard and sourced. I > ran a nova list after that: > No handlers could be found for logger > "keystoneclient.auth.identity.generic.base" > ERROR (Forbidden): Forbidden (HTTP 403) > > It still gives the error of forbidden access. > I think the password is not the issue. Forbidden access might be something > else. Do you want me to share anything else? > > On Wed, Apr 27, 2016 at 2:56 PM, Jens Rosenboom > wrote: > >> 2016-04-27 10:30 GMT+02:00 Dhvanan Shah : >> > UPDATE: >> > I am able to log into Horizon and perform all actions without any issue >> but >> > on my terminal, I am not able to do the same. The password that I >> thought >> > was wrong is not the issue as I logged in with the same password. >> > My keystone_adminrc file looks like this: >> > >> > unset OS_SERVICE_TOKEN OS_SERVICE_ENDPOINT >> > export OS_USERNAME=admin >> > export OS_PASSWORD= >> > export OS_AUTH_URL=http://10.16.37.221:35357/v2.0 >> > export PS1='[\u@\h \W(keystone_admin)]\$ ' >> > >> > export OS_TENANT_NAME=admin >> > export OS_REGION_NAME=RegionOne >> > >> > >> > Please suggest what I could do! >> >> Does your password contain special characters that might get mangled >> by the shell? You could compare the output of "echo $OS_PASSWORD" to >> verify. >> >> Otherwise, if the dashboard is working for you, you can go to >> Project/Compute/Access&Security/API Access and use the "Download >> OpenStack RC File" link there. >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > > -- > Dhvanan Shah > -- Dhvanan Shah __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Keystone Authorization Failed: Forbidden (HTTP 403)
Hi Jens, The password is correct when I echo $OS_PASSWORD. I downloaded the admin-openrc.sh file from the dashboard and sourced. I ran a nova list after that: No handlers could be found for logger "keystoneclient.auth.identity.generic.base" ERROR (Forbidden): Forbidden (HTTP 403) It still gives the error of forbidden access. I think the password is not the issue. Forbidden access might be something else. Do you want me to share anything else? On Wed, Apr 27, 2016 at 2:56 PM, Jens Rosenboom wrote: > 2016-04-27 10:30 GMT+02:00 Dhvanan Shah : > > UPDATE: > > I am able to log into Horizon and perform all actions without any issue > but > > on my terminal, I am not able to do the same. The password that I thought > > was wrong is not the issue as I logged in with the same password. > > My keystone_adminrc file looks like this: > > > > unset OS_SERVICE_TOKEN OS_SERVICE_ENDPOINT > > export OS_USERNAME=admin > > export OS_PASSWORD= > > export OS_AUTH_URL=http://10.16.37.221:35357/v2.0 > > export PS1='[\u@\h \W(keystone_admin)]\$ ' > > > > export OS_TENANT_NAME=admin > > export OS_REGION_NAME=RegionOne > > > > > > Please suggest what I could do! > > Does your password contain special characters that might get mangled > by the shell? You could compare the output of "echo $OS_PASSWORD" to > verify. > > Otherwise, if the dashboard is working for you, you can go to > Project/Compute/Access&Security/API Access and use the "Download > OpenStack RC File" link there. > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Dhvanan Shah __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Keystone Authorization Failed: Forbidden (HTTP 403)
UPDATE: I am able to log into Horizon and perform all actions without any issue but on my terminal, I am not able to do the same. The password that I thought was wrong is not the issue as I logged in with the same password. My keystone_adminrc file looks like this: unset OS_SERVICE_TOKEN OS_SERVICE_ENDPOINT export OS_USERNAME=admin export OS_PASSWORD= export OS_AUTH_URL=http://10.16.37.221:35357/v2.0 export PS1='[\u@\h \W(keystone_admin)]\$ ' export OS_TENANT_NAME=admin export OS_REGION_NAME=RegionOne Please suggest what I could do! On Wed, Apr 27, 2016 at 1:41 PM, Dhvanan Shah wrote: > All the services are running properly, it is just that the any action > performed says I am not authenticated or Forbidden (403) which means that > there is an authorization problem. In my keystone_adminrc file I have > exported all the environment variables and also set the admin password. Can > it be the case that this admin password is wrong because of which it is not > authenticating the admin right. If so how can I change the admin password? > > On Wed, Apr 27, 2016 at 12:05 PM, Dhvanan Shah wrote: > >> http://10.16.37.221:35357/v2.0/tokens/34e4b79b157a4526bc8ebb80b82cbf62 >> >> This link corresponding to the above token shows this error message - >> >> {"error": {"message": "The request you have made requires authentication.", >> "code": 401, "title": "Unauthorized"}} >> >> >> So there is a problem in authentication from the keystone.I haven't changed >> the admin password. >> >> >> On Wed, Apr 27, 2016 at 12:01 PM, Dhvanan Shah wrote: >> >>> Hi, >>> >>> I have a 2 node OpenStack cluster setup running on CentOS. Due to some >>> reason now I'm unable to perform any actions as it is not able to authorize >>> me, it shows an error message saying "Authorization Failed: Forbidden (HTTP >>> 403)" for Keystone when I run the command openstack-status and for >>> nova-managed services it says "No handlers could be found for logger >>> "keystoneclient.auth.identity.generic.base" >>> ERROR (Forbidden): Forbidden (HTTP 403)". >>> >>> The keystone logs at /var/log/keystone show no problem that might have >>> occured. >>> The nova api logs show this : >>> 2016-04-27 04:18:31.279 4314 DEBUG keystoneclient.session [-] REQ: curl >>> -g -i -X GET >>> http://10.16.37.221:35357/v2.0/tokens/34e4b79b157a4526bc8ebb80b82cbf62 >>> -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H >>> "X-Auth-Token: {SHA1}9d202e0a726633f660321097015cd9d67ac4df19" >>> _http_log_request >>> /usr/lib/python2.7/site-packages/keystoneclient/session.py:195 >>> 2016-04-27 04:18:31.290 4314 DEBUG keystoneclient.session [-] RESP: >>> _http_log_response >>> /usr/lib/python2.7/site-packages/keystoneclient/session.py:224 >>> 2016-04-27 04:18:31.290 4314 DEBUG keystoneclient.session [-] Request >>> returned failure status: 404 request >>> /usr/lib/python2.7/site-packages/keystoneclient/session.py:396 >>> 2016-04-27 04:18:31.290 4314 WARNING keystonemiddleware.auth_token [-] >>> Authorization failed for token >>> 2016-04-27 04:18:31.290 4314 WARNING keystonemiddleware.auth_token [-] >>> Identity response: {"error": {"message": "Could not find token: >>> 34e4b79b157a4526bc8ebb80b82cbf62", "code": 404, "title": "Not Found"}} >>> 2016-04-27 04:18:31.290 4314 WARNING keystonemiddleware.auth_token [-] >>> Authorization failed for token >>> >>> >>> Could someone please help me out as to how I could debug this issue. >>> >>> Thanks, >>> Dhvanan Shah >>> >> >> >> >> -- >> Dhvanan Shah >> > > > > -- > Dhvanan Shah > -- Dhvanan Shah __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Keystone Authorization Failed: Forbidden (HTTP 403)
All the services are running properly, it is just that the any action performed says I am not authenticated or Forbidden (403) which means that there is an authorization problem. In my keystone_adminrc file I have exported all the environment variables and also set the admin password. Can it be the case that this admin password is wrong because of which it is not authenticating the admin right. If so how can I change the admin password? On Wed, Apr 27, 2016 at 12:05 PM, Dhvanan Shah wrote: > http://10.16.37.221:35357/v2.0/tokens/34e4b79b157a4526bc8ebb80b82cbf62 > > This link corresponding to the above token shows this error message - > > {"error": {"message": "The request you have made requires authentication.", > "code": 401, "title": "Unauthorized"}} > > > So there is a problem in authentication from the keystone.I haven't changed > the admin password. > > > On Wed, Apr 27, 2016 at 12:01 PM, Dhvanan Shah wrote: > >> Hi, >> >> I have a 2 node OpenStack cluster setup running on CentOS. Due to some >> reason now I'm unable to perform any actions as it is not able to authorize >> me, it shows an error message saying "Authorization Failed: Forbidden (HTTP >> 403)" for Keystone when I run the command openstack-status and for >> nova-managed services it says "No handlers could be found for logger >> "keystoneclient.auth.identity.generic.base" >> ERROR (Forbidden): Forbidden (HTTP 403)". >> >> The keystone logs at /var/log/keystone show no problem that might have >> occured. >> The nova api logs show this : >> 2016-04-27 04:18:31.279 4314 DEBUG keystoneclient.session [-] REQ: curl >> -g -i -X GET >> http://10.16.37.221:35357/v2.0/tokens/34e4b79b157a4526bc8ebb80b82cbf62 >> -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H >> "X-Auth-Token: {SHA1}9d202e0a726633f660321097015cd9d67ac4df19" >> _http_log_request >> /usr/lib/python2.7/site-packages/keystoneclient/session.py:195 >> 2016-04-27 04:18:31.290 4314 DEBUG keystoneclient.session [-] RESP: >> _http_log_response >> /usr/lib/python2.7/site-packages/keystoneclient/session.py:224 >> 2016-04-27 04:18:31.290 4314 DEBUG keystoneclient.session [-] Request >> returned failure status: 404 request >> /usr/lib/python2.7/site-packages/keystoneclient/session.py:396 >> 2016-04-27 04:18:31.290 4314 WARNING keystonemiddleware.auth_token [-] >> Authorization failed for token >> 2016-04-27 04:18:31.290 4314 WARNING keystonemiddleware.auth_token [-] >> Identity response: {"error": {"message": "Could not find token: >> 34e4b79b157a4526bc8ebb80b82cbf62", "code": 404, "title": "Not Found"}} >> 2016-04-27 04:18:31.290 4314 WARNING keystonemiddleware.auth_token [-] >> Authorization failed for token >> >> >> Could someone please help me out as to how I could debug this issue. >> >> Thanks, >> Dhvanan Shah >> > > > > -- > Dhvanan Shah > -- Dhvanan Shah __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Keystone Authorization Failed: Forbidden (HTTP 403)
http://10.16.37.221:35357/v2.0/tokens/34e4b79b157a4526bc8ebb80b82cbf62 This link corresponding to the above token shows this error message - {"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}} So there is a problem in authentication from the keystone.I haven't changed the admin password. On Wed, Apr 27, 2016 at 12:01 PM, Dhvanan Shah wrote: > Hi, > > I have a 2 node OpenStack cluster setup running on CentOS. Due to some > reason now I'm unable to perform any actions as it is not able to authorize > me, it shows an error message saying "Authorization Failed: Forbidden (HTTP > 403)" for Keystone when I run the command openstack-status and for > nova-managed services it says "No handlers could be found for logger > "keystoneclient.auth.identity.generic.base" > ERROR (Forbidden): Forbidden (HTTP 403)". > > The keystone logs at /var/log/keystone show no problem that might have > occured. > The nova api logs show this : > 2016-04-27 04:18:31.279 4314 DEBUG keystoneclient.session [-] REQ: curl -g > -i -X GET > http://10.16.37.221:35357/v2.0/tokens/34e4b79b157a4526bc8ebb80b82cbf62 -H > "User-Agent: python-keystoneclient" -H "Accept: application/json" -H > "X-Auth-Token: {SHA1}9d202e0a726633f660321097015cd9d67ac4df19" > _http_log_request > /usr/lib/python2.7/site-packages/keystoneclient/session.py:195 > 2016-04-27 04:18:31.290 4314 DEBUG keystoneclient.session [-] RESP: > _http_log_response > /usr/lib/python2.7/site-packages/keystoneclient/session.py:224 > 2016-04-27 04:18:31.290 4314 DEBUG keystoneclient.session [-] Request > returned failure status: 404 request > /usr/lib/python2.7/site-packages/keystoneclient/session.py:396 > 2016-04-27 04:18:31.290 4314 WARNING keystonemiddleware.auth_token [-] > Authorization failed for token > 2016-04-27 04:18:31.290 4314 WARNING keystonemiddleware.auth_token [-] > Identity response: {"error": {"message": "Could not find token: > 34e4b79b157a4526bc8ebb80b82cbf62", "code": 404, "title": "Not Found"}} > 2016-04-27 04:18:31.290 4314 WARNING keystonemiddleware.auth_token [-] > Authorization failed for token > > > Could someone please help me out as to how I could debug this issue. > > Thanks, > Dhvanan Shah > -- Dhvanan Shah __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Keystone Authorization Failed: Forbidden (HTTP 403)
Hi, I have a 2 node OpenStack cluster setup running on CentOS. Due to some reason now I'm unable to perform any actions as it is not able to authorize me, it shows an error message saying "Authorization Failed: Forbidden (HTTP 403)" for Keystone when I run the command openstack-status and for nova-managed services it says "No handlers could be found for logger "keystoneclient.auth.identity.generic.base" ERROR (Forbidden): Forbidden (HTTP 403)". The keystone logs at /var/log/keystone show no problem that might have occured. The nova api logs show this : 2016-04-27 04:18:31.279 4314 DEBUG keystoneclient.session [-] REQ: curl -g -i -X GET http://10.16.37.221:35357/v2.0/tokens/34e4b79b157a4526bc8ebb80b82cbf62 -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}9d202e0a726633f660321097015cd9d67ac4df19" _http_log_request /usr/lib/python2.7/site-packages/keystoneclient/session.py:195 2016-04-27 04:18:31.290 4314 DEBUG keystoneclient.session [-] RESP: _http_log_response /usr/lib/python2.7/site-packages/keystoneclient/session.py:224 2016-04-27 04:18:31.290 4314 DEBUG keystoneclient.session [-] Request returned failure status: 404 request /usr/lib/python2.7/site-packages/keystoneclient/session.py:396 2016-04-27 04:18:31.290 4314 WARNING keystonemiddleware.auth_token [-] Authorization failed for token 2016-04-27 04:18:31.290 4314 WARNING keystonemiddleware.auth_token [-] Identity response: {"error": {"message": "Could not find token: 34e4b79b157a4526bc8ebb80b82cbf62", "code": 404, "title": "Not Found"}} 2016-04-27 04:18:31.290 4314 WARNING keystonemiddleware.auth_token [-] Authorization failed for token Could someone please help me out as to how I could debug this issue. Thanks, Dhvanan Shah __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Dynamically adding Extra Specs
Hey Jay! I was looking at implementing a few scheduling algorithms of my own natively into OpenStack, and for that I went through the nova-scheduler. After going through the scheduler, I felt that it was not very easy to implement or extend and add new scheduling algorithms to the scheduler. The only things that I felt that I could change or maybe was provisioned for adding or extending were the filters and weighers and implementing new scheduling algorithms with just these 2 knobs was a little hard. I did change the code in the filter_scheduler to get some basic algorithms running like the first and next fit apart from the spreading and stacking which was already present. But to go beyond and to implement more complex algorithms was much harder and I would have to change a lot of code in different places that could as a side effect also break things and didn't seem clean. I might be wrong and might have not understood things right, please correct me if so. To give an example of what I mean by a little complex scheduling algorithms: a subset matching algorithm - that schedules multiple heterogeneous requests by picking out a subset from the requests that best fit a host/s, so this would improve the utilization. The prerequisite for this is that I have multiple heterogeneous requests lined up to be scheduled. So for this kind of an algorithm it isnt easy to implement into OpenStack. So a workaround that I'm working on for implementing different scheduling algorithms is by building a scheduling wrapper outside of the OpenStack architecture, where the user interacts with this wrapper and in the wrapper I get the host details from the database and based on the algorithm I want, the scheduler chooses the host for the request and gives out a VM : Host mapping (The wrapper does the sanity checks that the filters do to check if the host can accommodate or handle the request). Along with the request, I also want to pass this mapping that the scheduler can use to assign the request to the host passed in the mapping. I've written a filter that filters all the hosts apart from the host that I sent and this is how I make sure that the request gets placed on the host that I had passed. I have come up with a hack to pass the host to the scheduler, but it is not quite elegant. Would be great to have your input on the same! On Mon, Feb 8, 2016 at 12:51 AM, Jay Pipes wrote: > Apologies for the delayed responses. Comments inline. > > On 01/27/2016 02:29 AM, Dhvanan Shah wrote: > >> Hey Jay! >> >> Thanks for the clarification. There was another thing that I wanted to >> know, is there any provision to pass extra arguments or some extra >> specifications along with the VM request to nova. To give you some >> context, I wanted to pass a host:vm mapping to the nova scheduler for >> its host selection process, and I'm providing this mapping from outside >> of the openstack architecture. >> > > Why do you want to do this? The scheduler is the thing that sets the host > -> vm mapping -- that's what the process of scheduling does. > > > So I need to send this information along > >> with the request to the scheduler. One way of doing this was creating >> new flavors with their extra specification as different hosts, but that >> would lead to as you pointed out earlier a "flavor explosion" problem. >> >> So is there a way to pass some extra arguments or some additional >> information to nova. >> > > Depends what exactly you are trying to pass to Nova. Could you give some > more information about your use case? > > Thanks! > -jay > -- Dhvanan Shah __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Dynamically adding Extra Specs
Hey Jay! Thanks for the clarification. There was another thing that I wanted to know, is there any provision to pass extra arguments or some extra specifications along with the VM request to nova. To give you some context, I wanted to pass a host:vm mapping to the nova scheduler for its host selection process, and I'm providing this mapping from outside of the openstack architecture. So I need to send this information along with the request to the scheduler. One way of doing this was creating new flavors with their extra specification as different hosts, but that would lead to as you pointed out earlier a "flavor explosion" problem. So is there a way to pass some extra arguments or some additional information to nova. On Fri, Jan 22, 2016 at 7:17 PM, Jay Pipes wrote: > Hi! Comments inline... > > On 01/22/2016 01:26 AM, Dhvanan Shah wrote: > >> Hi, >> >> I had a few queries regarding adding extra specs for VM requests. >> >> According to my understanding if I want to add extra specs to requests >> then I need to change that in different flavors adding those >> capabilities by setting them in the flavors. But if the requests that I >> get have varying values for those extra capabilities then it seems to >> create an issue as the values in the flavors are static. Please correct >> me if I'm wrong. >> > > Right, flavors are static blobs of both resources (amounts of things > requested by the user) and capabilities (free-form key/value pairs in > extra_specs). For every variation you may want to offer your cloud users, > you need to create a new flavor in the system, and your cloud users must > select that flavor when booting a VM. > > Don Dugger from Intel has called this problem "flavor explosion", which I > have comically re-labeled "the Skittles problem". We have a long-term plan > to allow for the ability to list a set of capabilities present in the > deployment and allow a cloud user to "mix and match" resource amounts and > required capabilities in a more flexible way, but that work is likely 9-12 > months out in reality. > > So I wanted to know as to how I could dynamically add those extra specs >> best suiting each request. Is there a way of mentioning the extra specs >> everytime I spawn a VM through the nova cli? Setting and unsetting the >> extra specs everytime I spawn VM's according to my need would be quite >> inefficient as it makes changes to the database. >> > > Yes, it is a clunky and inflexible API right now, I agree. The genesis of > the flavor concept comes from Rackspace Cloud Servers, which modeled its > flavor concept on the Amazon EC2 instance types concept. This model works > well for public clouds, because a) they use the flavor as a stock-keeping > unit (SKU) so therefore this system makes billing easier, and b) it reduces > the number of hardware configuration options that public cloud operators > need to provide at scale to users. > > If you are interested in this area, follow conversations and blueprints > with the terms "flavor decomposition", "host capabilities", and "resource > representation". > > Best, > -jay > > __________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Dhvanan Shah __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Dynamically adding Extra Specs
Hi, I had a few queries regarding adding extra specs for VM requests. According to my understanding if I want to add extra specs to requests then I need to change that in different flavors adding those capabilities by setting them in the flavors. But if the requests that I get have varying values for those extra capabilities then it seems to create an issue as the values in the flavors are static. Please correct me if I'm wrong. So I wanted to know as to how I could dynamically add those extra specs best suiting each request. Is there a way of mentioning the extra specs everytime I spawn a VM through the nova cli? Setting and unsetting the extra specs everytime I spawn VM's according to my need would be quite inefficient as it makes changes to the database. Thanks, Dhvanan Shah __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] ERROR : openstack Forbidden (HTTP 403)
Hi, I have not been able to resolve it. The problem of "OpenStack Forbidden (HTTP 403) still persists. ERROR : Error appeared during Puppet run: 10.16.37.221_keystone.pp Error: /Stage[main]/Neutron::Keystone::Auth/Keystone::Resource::Service_identity[neutron]/Keystone_user[neutron]: Could not evaluate: Execution of '/usr/bin/openstack token issue --format value' returned 1: WARNING: keystoneclient.auth.identity.generic.base Discovering versions from the identity service failed when creating the password plugin. Attempting to determine version from URL. You will find full trace in log /var/tmp/packstack/20151015-212559-xwC1zD/manifests/10.16.37.221_keystone.pp.log Could someone please help me with this? On Mon, Oct 12, 2015 at 5:16 PM, Dhvanan Shah wrote: > Resolved it. > The no_proxy env var needs to be set if your computer is located behind a > authenticating proxy infrastructure. > > Source : > > https://ask.openstack.org/en/question/67203/kilo-deployment-using-packstack-fails-with-403-error-on-usrbinopenstack-service-list/ > > On Mon, Oct 12, 2015 at 3:12 PM, Dhvanan Shah wrote: > >> Hi, >> >> I am getting this error while installing Openstack on Centos. >> ERROR : Error appeared during Puppet run: 10.16.37.221_keystone.pp >> Error: Could not prefetch keystone_service provider 'openstack': >> Execution of '/usr/bin/openstack service list --quiet --format csv --long' >> returned 1: ERROR: openstack Forbidden (HTTP 403) >> >> I've checked the permissions of the the executable files and they are not >> the problem. So I'm not sure why I'm forbidden from executing this. Could >> use some help! >> >> -- >> Dhvanan Shah >> > > > > -- > Dhvanan Shah > -- Dhvanan Shah __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] ERROR : openstack Forbidden (HTTP 403)
Resolved it. The no_proxy env var needs to be set if your computer is located behind a authenticating proxy infrastructure. Source : https://ask.openstack.org/en/question/67203/kilo-deployment-using-packstack-fails-with-403-error-on-usrbinopenstack-service-list/ On Mon, Oct 12, 2015 at 3:12 PM, Dhvanan Shah wrote: > Hi, > > I am getting this error while installing Openstack on Centos. > ERROR : Error appeared during Puppet run: 10.16.37.221_keystone.pp > Error: Could not prefetch keystone_service provider 'openstack': Execution > of '/usr/bin/openstack service list --quiet --format csv --long' returned > 1: ERROR: openstack Forbidden (HTTP 403) > > I've checked the permissions of the the executable files and they are not > the problem. So I'm not sure why I'm forbidden from executing this. Could > use some help! > > -- > Dhvanan Shah > -- Dhvanan Shah __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] ERROR : openstack Forbidden (HTTP 403)
Hi, I am getting this error while installing Openstack on Centos. ERROR : Error appeared during Puppet run: 10.16.37.221_keystone.pp Error: Could not prefetch keystone_service provider 'openstack': Execution of '/usr/bin/openstack service list --quiet --format csv --long' returned 1: ERROR: openstack Forbidden (HTTP 403) I've checked the permissions of the the executable files and they are not the problem. So I'm not sure why I'm forbidden from executing this. Could use some help! -- Dhvanan Shah __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] (NOVA) Is there a Queue maintained for instance requests
Hi, I had a question regarding the process of spawning an instance. In the whole process from requesting an instance to scheduling to spawning ,is there a queue maintained where the requests go first, like in the case of a scheduler of any OS that has a queue for jobs to be scheduled. This question arises as I wanted to look at handling multiple instance requests at a time and wanted to see if there was a common place where all the instances get registered first and get spawned after that. I went through the code base and tried to find if there was a queue. I tried to traceback from the side of the client -> create( in nova/api/openstack/compute/servers.py ) api server that handles all the requests ->create and _create_instance (in nova/compute/api.py) that handle all the requests regarding the compute resources. ->build_instances (nova/conductor/manager.py) handles all the db operations from here on the request is sent to the scheduler that return a host for the instance. So I'm not sure if I missed it but I was not able to find any queue where the requests are being registered. Could someone please help me understand how this works. Cheers! -- Dhvanan Shah __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Tracing a request (NOVA)
Hi, I'm trying to trace a request made for an instance and looking at the flow in the code. I'm just trying to understand better how the request goes from the dashboard to the nova-api , to the other internal components of nova and to the scheduler and back with a suitable host and launching of the instance. i just want to understand as to how the request goes from the api-call to the nova-api and so on after that. I have understood the nova-scheduler and in that, the filter_scheduler receives something called request_spec that is the specifications of the request that is made, and I want to see where this comes from. I was not very successful in reverse engineering this. I could use some help as I want to implement a scheduling algorithm of my own but for that I need to understand how and where the requests come in and how the flow works. If someone could guide me as to where i can find help or point in some direction then it would be of great help. -- Dhvanan Shah __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev