Keystone signs the information in auth token with a certificate that in most setups was generated for that instance of keystone. Swift will use auth_token middleware to fetch the certificates of keystone so that it can verify that the tokens are correct.
My guess is that the two keystone instances are using different certificates and you are trying to validate a token with the other keystone instance (other certificates) and it won't work. If you are using the same keystone instance then it is possible that the auth_token middleware in swift has cached the certificates for the other keystone instance, so even though you have updated the values in swift it is using the old certificates. Try deleting the certificates found in the folder specified by signing_dir in the swift setup and make sure you are issuing the tokens from the keystone instance you are validating them against. Jamie On Wed, 2013-10-30 at 18:47 +0100, thorf...@poivron.org wrote: > Hi all. > > * Hypervisor 1 : 192.168.1.120 > - Keystone 1 : 192.168.3.141 > - Swift-proxy 1 : 192.168.3.111 > * Hypervisor 2 : 192.168.1.122 > - Keystone 2 : 192.168.3.241 > - Swift-proxy 2 : 192.168.3.211 > > Keystone servers have the same mysql server, database and > configuration, so it's not a data issue. > Every server can ping and talk to all the other ones. > > When I talk to Swift-proxy 1, connected to Keystone 1 it works. > Same to Swift-proxy 2, connected to Keystone 2. > > If I connect Swift-proxy 1 to Keystone 2, it doesn't work anymore. > Same for Swift-proxy 2 to Keystone 1. > > All the servers are using Ubuntu 12.04.3/Havana and are up-to-date. > > When it works, I have this (keystone 2 connected to swift-proxy 2) : > # swift -V 2 -v -A http://192.168.3.241:5000/v2.0 -U service:swift -K > swift stat > StorageURL: > http://192.168.3.211:8080/v1/AUTH_5becb4a93e7f498bbe83534f4481dc0d > Auth Token: > MIIGTQYJKoZIhvcNAQcCoIIGPjCCBjoCAQExCTAHBgUrDgMCGjCCBKMGCSqGSIb3DQEHAaCCBJQEggSQeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxMy0xMC0zMFQxNzozMzowMC4zNTI4MzAiLCAiZXhwaXJlcyI6ICIyMDEzLTEwLTMxVDE3OjMzOjAwWiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogIlNlcnZpY2UgVGVuYW50IiwgImVuYWJsZWQiOiB0cnVlLCAiaWQiOiAiNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAibmFtZSI6ICJzZXJ2aWNlIn19LCAic2VydmljZUNhdGFsb2ciOiBbeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjIxMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjIxMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAiaWQiOiAiMzdhZWQxODc3NmQ5NDUzZGI3MmE3ODc1ZWM3ZTY5ZGEiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjExOjgwODAvdjEvQVVUSF81YmVjYjRhOTNlN2Y0OThiYmU4MzUzNGY0NDgxZGMwZCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJvYmplY3Qtc3RvcmUiLCAibmFtZSI6ICJzd2lmdCJ9LCB7ImVuZHBvaW50cyI6IFt7ImF! kbW > luVVJMIj > ogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjM1MzU3L3YyLjAiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjI0MTo1MDAwL3YyLjAiLCAiaWQiOiAiMGJmNzIxNjM1MmFjNDE4ZmEzODVkNWZmM2ZmODlmMzAiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjUwMDAvdjIuMCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJpZGVudGl0eSIsICJuYW1lIjogImtleXN0b25lIn1dLCAidXNlciI6IHsidXNlcm5hbWUiOiAic3dpZnQiLCAicm9sZXNfbGlua3MiOiBbXSwgImlkIjogIjYzZWVjNjMyYWI3NTRiYzY5NTgzY2M0YTI5Yjc0MDVmIiwgInJvbGVzIjogW3sibmFtZSI6ICJhZG1pbiJ9XSwgIm5hbWUiOiAic3dpZnQifSwgIm1ldGFkYXRhIjogeyJpc19hZG1pbiI6IDAsICJyb2xlcyI6IFsiZjUwNjczYjliNDUwNDQyZGI1OGRkYTExYjQ4M2ZkMmUiXX19fTGCAYEwggF9AgEBMFwwVzELMAkGA1UEBhMCVVMxDjAMBgNVBAgMBVVuc2V0MQ4wDAYDVQQHDAVVbnNldDEOMAwGA1UECgwFVW5zZXQxGDAWBgNVBAMMD3d3dy5leGFtcGxlLmNvbQIBATAHBgUrDgMCGjANBgkqhkiG9w0BAQEFAASCAQASAXjP5gHiUWfl0e8gfh2O4n7IEoerLDmmTR07tQw1ibqyxyyhdNAVuThrceu0z3-VrHyeiuYcWAlJZzI6okeo02CDc7SRK4CjHLG9m8q80UrLOfT1-PbKes16ULKbGJpsBYykVXTV8ts+wQVAYcS73f2bwp6+Ki0Cygtfqkmvq! Net > 7hDtSsvT > yAGKiLo0TbOYOpF96NelPwuzGAm2y-bcOhCCdJKo8iFEotTXK0SQzUQ78r3Mh1fsd6asoHRZxKKc0oXWm3KgJy1X-isnqopCMutDPPQCAXABOFb-OSovLMmmOS8ZZbII7RTd1e1z1sFYv3d67b0oc2A4e8DWAaVj > Account: AUTH_5becb4a93e7f498bbe83534f4481dc0d > Containers: 4 > Objects: 11 > Bytes: 158989835 > Accept-Ranges: bytes > X-Timestamp: 1382628587.87452 > Content-Type: text/plain; charset=utf-8 > > Oct 30 18:32:59 dev-api-002 proxy-server Verify error: Command > 'openssl' returned non-zero exit status 4 > Oct 30 18:32:59 dev-api-002 proxy-server Authorization failed for token > MIIGTQYJKoZIhvcNAQcCoIIGPjCCBjoCAQExCTAHBgUrDgMCGjCCBKMGCSqGSIb3DQEHAaCCBJQEggSQeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMj > AxMy0xMC0zMFQxNzozMjo1OC44NTY3MzEiLCAiZXhwaXJlcyI6ICIyMDEzLTEwLTMxVDE3OjMyOjU4WiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogIlNlcnZpY2UgVGVuYW50IiwgImVuYWJsZWQiOiB0cnVlLCAiaWQiOiAiNWJl > Y2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAibmFtZSI6ICJzZXJ2aWNlIn19LCAic2VydmljZUNhdGFsb2ciOiBbeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjIxMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlOD > M1MzRmNDQ4MWRjMGQiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjIxMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAiaWQiOiAiMzdhZWQxODc3NmQ5NDUzZGI3MmE3ODc1 > ZWM3ZTY5ZGEiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjExOjgwODAvdjEvQVVUSF81YmVjYjRhOTNlN2Y0OThiYmU4MzUzNGY0NDgxZGMwZCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJvYmplY3Qtc3RvcmUiLCAibmFtZSI6ICJzd2 > lmdCJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjM1MzU3L3YyLjAiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjI0MTo1MDAwL3YyLjAiLCAiaWQiOiAiMGJmNzIx > NjM1MmFjNDE4ZmEzODVkNWZmM2ZmODlmMzAiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjUwMDAvdjIuMCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJpZGVudGl0eSIsICJuYW1lIjogImtleXN0b25lIn1dLCAidXNlciI6IHsidX > Nlcm5hbWUiOiAic3dpZnQiLCAicm9sZXNfbGlua3MiOiBbXSwgImlkIjogIjYzZWVjNjMyYWI3NTRiYzY5NTgzY2M0YTI5Yjc0MDVmIiwgInJvbGVzIjogW3sibmFtZSI6ICJhZG1pbiJ9XSwgIm5hbWUiOiAic3dpZnQifSwgIm1ldGFkYXRhIjogeyJpc19hZG1pbiI6IDAs > ICJyb2xlcyI6IFsiZjUwNjczYjliNDUwNDQyZGI1OGRkYTExYjQ4M2ZkMmUiXX19fTGCAYEwggF9AgEBMFwwVzELMAkGA1UEBhMCVVMxDjAMBgNVBAgMBVVuc2V0MQ4wDAYDVQQHDAVVbnNldDEOMAwGA1UECgwFVW5zZXQxGDAWBgNVBAMMD3d3dy5leGFtcGxlLmNvbQIBAT > AHBgUrDgMCGjANBgkqhkiG9w0BAQEFAASCAQCzitgoJ4ltBsCNN8xnLy3GlopgV5OlVRBa4fbHXcNT6expAdTYtx4I8q1cIF279NPVJO9T8hsedMSHwOxZvxJKskwFuuwUWT+cTBzkxlrY11Njmg9dGwQiJ1Pbb8oA3YZcgWjz6aY+1RajN-Lq9ugCidsY5tzFrHTwPed1VOcu > Wq2MKcMIqmt2m5b > Oct 30 18:32:59 dev-api-002 proxy-server Invalid user token - deferring > reject downstream > > Why the error if it works ? > > When it doesn't work, I have this (keystone 2 connected to swift-proxy > 1) : > # swift -V 2 -v -A http://192.168.3.241:5000/v2.0 -U service:swift -K > swift stat > Account HEAD failed: > http://192.168.3.111:8080/v1/AUTH_5becb4a93e7f498bbe83534f4481dc0d 401 > Unauthorized > > Oct 30 18:34:53 dev-api-001 proxy-server Verify error: Command > 'openssl' returned non-zero exit status 4 > Oct 30 18:34:53 dev-api-001 proxy-server Authorization failed for token > MIIGTQYJKoZIhvcNAQcCoIIGPjCCBjoCAQExCTAHBgUrDgMCGjCCBKMGCSqGSIb3DQEHAaCCBJQEggSQeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMj > AxMy0xMC0zMFQxNzozNDo1My42NTY0NTMiLCAiZXhwaXJlcyI6ICIyMDEzLTEwLTMxVDE3OjM0OjUzWiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogIlNlcnZpY2UgVGVuYW50IiwgImVuYWJsZWQiOiB0cnVlLCAiaWQiOiAiNWJl > Y2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAibmFtZSI6ICJzZXJ2aWNlIn19LCAic2VydmljZUNhdGFsb2ciOiBbeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjExMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlOD > M1MzRmNDQ4MWRjMGQiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjExMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAiaWQiOiAiMzdhZWQxODc3NmQ5NDUzZGI3MmE3ODc1 > ZWM3ZTY5ZGEiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMTExOjgwODAvdjEvQVVUSF81YmVjYjRhOTNlN2Y0OThiYmU4MzUzNGY0NDgxZGMwZCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJvYmplY3Qtc3RvcmUiLCAibmFtZSI6ICJzd2 > lmdCJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjM1MzU3L3YyLjAiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjI0MTo1MDAwL3YyLjAiLCAiaWQiOiAiMGJmNzIx > NjM1MmFjNDE4ZmEzODVkNWZmM2ZmODlmMzAiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjUwMDAvdjIuMCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJpZGVudGl0eSIsICJuYW1lIjogImtleXN0b25lIn1dLCAidXNlciI6IHsidX > Nlcm5hbWUiOiAic3dpZnQiLCAicm9sZXNfbGlua3MiOiBbXSwgImlkIjogIjYzZWVjNjMyYWI3NTRiYzY5NTgzY2M0YTI5Yjc0MDVmIiwgInJvbGVzIjogW3sibmFtZSI6ICJhZG1pbiJ9XSwgIm5hbWUiOiAic3dpZnQifSwgIm1ldGFkYXRhIjogeyJpc19hZG1pbiI6IDAs > ICJyb2xlcyI6IFsiZjUwNjczYjliNDUwNDQyZGI1OGRkYTExYjQ4M2ZkMmUiXX19fTGCAYEwggF9AgEBMFwwVzELMAkGA1UEBhMCVVMxDjAMBgNVBAgMBVVuc2V0MQ4wDAYDVQQHDAVVbnNldDEOMAwGA1UECgwFVW5zZXQxGDAWBgNVBAMMD3d3dy5leGFtcGxlLmNvbQIBAT > AHBgUrDgMCGjANBgkqhkiG9w0BAQEFAASCAQAZRHKSulq-73euRy9HrYxMTU-HtRizyySVYkoo3CTCOgxFZz3CzelBIcp6HJySC6DVAW4Uz-xcTmtp1hju3vx3yAVstWtCczO-YZX4bUy4XFmfNje2ydJl5M2sSAUZ8160Vn3QnajesaRIvnu9w8WcpWsmaYjbx15ou2CzWnvH > j0V1lLTgA28dh90 > Oct 30 18:34:53 dev-api-001 proxy-server Invalid user token - deferring > reject downstream > Oct 30 18:34:55 dev-api-001 proxy-server Verify error: Command > 'openssl' returned non-zero exit status 4 > Oct 30 18:34:55 dev-api-001 proxy-server Authorization failed for token > MIIGTQYJKoZIhvcNAQcCoIIGPjCCBjoCAQExCTAHBgUrDgMCGjCCBKMGCSqGSIb3DQEHAaCCBJQEggSQeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMj > AxMy0xMC0zMFQxNzozNDo1NS4xNTA5MjUiLCAiZXhwaXJlcyI6ICIyMDEzLTEwLTMxVDE3OjM0OjU1WiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogIlNlcnZpY2UgVGVuYW50IiwgImVuYWJsZWQiOiB0cnVlLCAiaWQiOiAiNWJl > Y2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAibmFtZSI6ICJzZXJ2aWNlIn19LCAic2VydmljZUNhdGFsb2ciOiBbeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjExMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlOD > M1MzRmNDQ4MWRjMGQiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjExMTo4MDgwL3YxL0FVVEhfNWJlY2I0YTkzZTdmNDk4YmJlODM1MzRmNDQ4MWRjMGQiLCAiaWQiOiAiMzdhZWQxODc3NmQ5NDUzZGI3MmE3ODc1 > ZWM3ZTY5ZGEiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMTExOjgwODAvdjEvQVVUSF81YmVjYjRhOTNlN2Y0OThiYmU4MzUzNGY0NDgxZGMwZCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJvYmplY3Qtc3RvcmUiLCAibmFtZSI6ICJzd2 > lmdCJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjM1MzU3L3YyLjAiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTkyLjE2OC4zLjI0MTo1MDAwL3YyLjAiLCAiaWQiOiAiMGJmNzIx > NjM1MmFjNDE4ZmEzODVkNWZmM2ZmODlmMzAiLCAicHVibGljVVJMIjogImh0dHA6Ly8xOTIuMTY4LjMuMjQxOjUwMDAvdjIuMCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJpZGVudGl0eSIsICJuYW1lIjogImtleXN0b25lIn1dLCAidXNlciI6IHsidX > Nlcm5hbWUiOiAic3dpZnQiLCAicm9sZXNfbGlua3MiOiBbXSwgImlkIjogIjYzZWVjNjMyYWI3NTRiYzY5NTgzY2M0YTI5Yjc0MDVmIiwgInJvbGVzIjogW3sibmFtZSI6ICJhZG1pbiJ9XSwgIm5hbWUiOiAic3dpZnQifSwgIm1ldGFkYXRhIjogeyJpc19hZG1pbiI6IDAs > ICJyb2xlcyI6IFsiZjUwNjczYjliNDUwNDQyZGI1OGRkYTExYjQ4M2ZkMmUiXX19fTGCAYEwggF9AgEBMFwwVzELMAkGA1UEBhMCVVMxDjAMBgNVBAgMBVVuc2V0MQ4wDAYDVQQHDAVVbnNldDEOMAwGA1UECgwFVW5zZXQxGDAWBgNVBAMMD3d3dy5leGFtcGxlLmNvbQIBAT > AHBgUrDgMCGjANBgkqhkiG9w0BAQEFAASCAQCXUDhH+Q8xUS6upUb8TtF2Uk-h2m-w84CmeJxKc-n7qGuozrZe7KPcrKp002ojDIY+CmGulWtXQD-IJ6V4hcjaVbmoxMIIVmMulxt1G2dLLIrtQCIUwnNFsOaaBiEZTus8DlpjIHGrLfcBRtzjewQXUA5PuRXC-ebtgE7wphMv > ETodRWB5zKixqmL > Oct 30 18:34:55 dev-api-001 proxy-server Invalid user token - deferring > reject downstream > > I have the same kind of logs entry than the working example but twice. > > _______________________________________________ > Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > Post to : openstack@lists.openstack.org > Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack _______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack