Re: [Openstack-operators] [Octavia] SSL errors polling amphorae and missing tenant network interface
Hi eric!
Glad I’m not the only one having this issue with the ssl communication
between the amphora and the CP.
Even if I don’t yet get a clear answer regarding that issue, I think your
second issue is not an issue as the interface is mounted on a namespace and
so you’ll need to list all nic even those from namespace.
Use an ip netns ls to get the namespace.
Hope it will help.
Le ven. 19 oct. 2018 à 20:40, Erik McCormick a
écrit :
> I've been wrestling with getting Octavia up and running and have
> become stuck on two issues. I'm hoping someone has run into these
> before. My google foo has come up empty.
>
> Issue 1:
> When the Octavia controller tries to poll the amphora instance, it
> tries repeatedly and eventually fails. The error on the controller
> side is:
>
> 2018-10-19 14:17:39.181 26 ERROR
> octavia.amphorae.drivers.haproxy.rest_api_driver [-] Connection
> retries (currently set to 300) exhausted. The amphora is unavailable.
> Reason: HTTPSConnectionPool(host='10.7.0.112', port=9443): Max retries
> exceeded with url: /0.5/plug/vip/10.250.20.15 (Caused by
> SSLError(SSLError("bad handshake: Error([('rsa routines',
> 'RSA_padding_check_PKCS1_type_1', 'invalid padding'), ('rsa routines',
> 'rsa_ossl_public_decrypt', 'padding check failed'), ('asn1 encoding
> routines', 'ASN1_item_verify', 'EVP lib'), ('SSL routines',
> 'tls_process_server_certificate', 'certificate verify
> failed')],)",),)): SSLError: HTTPSConnectionPool(host='10.7.0.112',
> port=9443): Max retries exceeded with url: /0.5/plug/vip/10.250.20.15
> (Caused by SSLError(SSLError("bad handshake: Error([('rsa routines',
> 'RSA_padding_check_PKCS1_type_1', 'invalid padding'), ('rsa routines',
> 'rsa_ossl_public_decrypt', 'padding check failed'), ('asn1 encoding
> routines', 'ASN1_item_verify', 'EVP lib'), ('SSL routines',
> 'tls_process_server_certificate', 'certificate verify
> failed')],)",),))
>
> On the amphora side I see:
> [2018-10-19 17:52:54 +] [1331] [DEBUG] Error processing SSL request.
> [2018-10-19 17:52:54 +] [1331] [DEBUG] Invalid request from
> ip=:::10.7.0.40: [SSL: SSL_HANDSHAKE_FAILURE] ssl handshake
> failure (_ssl.c:1754)
>
> I've generated certificates both with the script in the Octavia git
> repo, and with the Openstack Ansible playbook. I can see that they are
> present in /etc/octavia/certs.
>
> I'm using the Kolla (Queens) containers for the control plane so I'm
> sure I've satisfied all the python library constraints.
>
> Issue 2:
> I"m not sure how it gets configured, but the tenant network interface
> (ens6) never comes up. I can spawn other instances on that network
> with no issue, and I can see that Neutron has the port attached to the
> instance. However, in the instance this is all I get:
>
> ubuntu@amphora-33e0aab3-8bc4-4fcb-bc42-b9b36afb16d4:~$ ip a
> 1: lo: mtu 65536 qdisc noqueue state UNKNOWN
> group default qlen 1
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> inet 127.0.0.1/8 scope host lo
>valid_lft forever preferred_lft forever
> inet6 ::1/128 scope host
>valid_lft forever preferred_lft forever
> 2: ens3: mtu 9000 qdisc pfifo_fast
> state UP group default qlen 1000
> link/ether fa:16:3e:30:c4:60 brd ff:ff:ff:ff:ff:ff
> inet 10.7.0.112/16 brd 10.7.255.255 scope global ens3
>valid_lft forever preferred_lft forever
> inet6 fe80::f816:3eff:fe30:c460/64 scope link
>valid_lft forever preferred_lft forever
> 3: ens6: mtu 1500 qdisc noop state DOWN group
> default qlen 1000
> link/ether fa:16:3e:89:a2:7f brd ff:ff:ff:ff:ff:ff
>
> There's no evidence of the interface anywhere else including udev rules.
>
> Any help with either or both issues would be greatly appreciated.
>
> Cheers,
> Erik
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
[Openstack-operators] [Octavia] SSL errors polling amphorae and missing tenant network interface
I've been wrestling with getting Octavia up and running and have
become stuck on two issues. I'm hoping someone has run into these
before. My google foo has come up empty.
Issue 1:
When the Octavia controller tries to poll the amphora instance, it
tries repeatedly and eventually fails. The error on the controller
side is:
2018-10-19 14:17:39.181 26 ERROR
octavia.amphorae.drivers.haproxy.rest_api_driver [-] Connection
retries (currently set to 300) exhausted. The amphora is unavailable.
Reason: HTTPSConnectionPool(host='10.7.0.112', port=9443): Max retries
exceeded with url: /0.5/plug/vip/10.250.20.15 (Caused by
SSLError(SSLError("bad handshake: Error([('rsa routines',
'RSA_padding_check_PKCS1_type_1', 'invalid padding'), ('rsa routines',
'rsa_ossl_public_decrypt', 'padding check failed'), ('asn1 encoding
routines', 'ASN1_item_verify', 'EVP lib'), ('SSL routines',
'tls_process_server_certificate', 'certificate verify
failed')],)",),)): SSLError: HTTPSConnectionPool(host='10.7.0.112',
port=9443): Max retries exceeded with url: /0.5/plug/vip/10.250.20.15
(Caused by SSLError(SSLError("bad handshake: Error([('rsa routines',
'RSA_padding_check_PKCS1_type_1', 'invalid padding'), ('rsa routines',
'rsa_ossl_public_decrypt', 'padding check failed'), ('asn1 encoding
routines', 'ASN1_item_verify', 'EVP lib'), ('SSL routines',
'tls_process_server_certificate', 'certificate verify
failed')],)",),))
On the amphora side I see:
[2018-10-19 17:52:54 +] [1331] [DEBUG] Error processing SSL request.
[2018-10-19 17:52:54 +] [1331] [DEBUG] Invalid request from
ip=:::10.7.0.40: [SSL: SSL_HANDSHAKE_FAILURE] ssl handshake
failure (_ssl.c:1754)
I've generated certificates both with the script in the Octavia git
repo, and with the Openstack Ansible playbook. I can see that they are
present in /etc/octavia/certs.
I'm using the Kolla (Queens) containers for the control plane so I'm
sure I've satisfied all the python library constraints.
Issue 2:
I"m not sure how it gets configured, but the tenant network interface
(ens6) never comes up. I can spawn other instances on that network
with no issue, and I can see that Neutron has the port attached to the
instance. However, in the instance this is all I get:
ubuntu@amphora-33e0aab3-8bc4-4fcb-bc42-b9b36afb16d4:~$ ip a
1: lo: mtu 65536 qdisc noqueue state UNKNOWN
group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens3: mtu 9000 qdisc pfifo_fast
state UP group default qlen 1000
link/ether fa:16:3e:30:c4:60 brd ff:ff:ff:ff:ff:ff
inet 10.7.0.112/16 brd 10.7.255.255 scope global ens3
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe30:c460/64 scope link
valid_lft forever preferred_lft forever
3: ens6: mtu 1500 qdisc noop state DOWN group
default qlen 1000
link/ether fa:16:3e:89:a2:7f brd ff:ff:ff:ff:ff:ff
There's no evidence of the interface anywhere else including udev rules.
Any help with either or both issues would be greatly appreciated.
Cheers,
Erik
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Fleio - OpenStack billing - ver. 1.1 released
On Fri, Oct 19, 2018 at 7:45 PM Jay Pipes wrote: > > Please do not use these mailing lists to advertise > closed-source/proprietary software solutions. +1 > Thank you, > -jay > > On 10/19/2018 05:42 AM, Adrian Andreias wrote: > > Hello, > > > > We've just released Fleio version 1.1. > > > > Fleio is a billing solution and control panel for OpenStack public > > clouds and traditional web hosters. > > > > Fleio software automates the entire process for cloud users. New > > customers can use Fleio to sign up for an account, pay invoices, add > > credit to their account, as well as create and manage cloud resources > > such as virtual machines, storage and networking. > > > > Full feature list: > > https://fleio.com#features > > > > You can see an online demo: > > https://fleio.com/demo > > > > And sign-up for a free trial: > > https://fleio.com/signup > > > > > > > > Cheers! > > > > - Adrian Andreias > > https://fleio.com > > > > > > > > ___ > > OpenStack-operators mailing list > > OpenStack-operators@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > > > > ___ > OpenStack-operators mailing list > OpenStack-operators@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators -- Mohammed Naser — vexxhost - D. 514-316-8872 D. 800-910-1726 ext. 200 E. mna...@vexxhost.com W. http://vexxhost.com ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Fleio - OpenStack billing - ver. 1.1 released
Please do not use these mailing lists to advertise closed-source/proprietary software solutions. Thank you, -jay On 10/19/2018 05:42 AM, Adrian Andreias wrote: Hello, We've just released Fleio version 1.1. Fleio is a billing solution and control panel for OpenStack public clouds and traditional web hosters. Fleio software automates the entire process for cloud users. New customers can use Fleio to sign up for an account, pay invoices, add credit to their account, as well as create and manage cloud resources such as virtual machines, storage and networking. Full feature list: https://fleio.com#features You can see an online demo: https://fleio.com/demo And sign-up for a free trial: https://fleio.com/signup Cheers! - Adrian Andreias https://fleio.com ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Glance Image Visibility Issue? - Non admin users can see private images from other tenants
For reference, here is our full glance policy.json
{
"context_is_admin": "role:admin",
"default": "role:admin",
"add_image": "",
"delete_image": "",
"get_image": "",
"get_images": "",
"modify_image": "",
"publicize_image": "role:admin",
"communitize_image": "",
"copy_from": "",
"download_image": "",
"upload_image": "",
"delete_image_location": "",
"get_image_location": "",
"set_image_location": "",
"add_member": "",
"delete_member": "",
"get_member": "",
"get_members": "",
"modify_member": "",
"manage_image_cache": "role:admin",
"get_task": "",
"get_tasks": "",
"add_task": "",
"modify_task": "",
"tasks_api_access": "role:admin",
"deactivate": "",
"reactivate": "",
"get_metadef_namespace": "",
"get_metadef_namespaces":"",
"modify_metadef_namespace":"",
"add_metadef_namespace":"",
"get_metadef_object":"",
"get_metadef_objects":"",
"modify_metadef_object":"",
"add_metadef_object":"",
"list_metadef_resource_types":"",
"get_metadef_resource_type":"",
"add_metadef_resource_type_association":"",
"get_metadef_property":"",
"get_metadef_properties":"",
"modify_metadef_property":"",
"add_metadef_property":"",
"get_metadef_tag":"",
"get_metadef_tags":"",
"modify_metadef_tag":"",
"add_metadef_tag":"",
"add_metadef_tags":""
}
Mike Moore, M.S.S.E.
Systems Engineer, Goddard Private Cloud
michael.d.mo...@nasa.gov
Hydrogen fusion brightens my day.
On 10/19/18, 12:39 PM, "Moore, Michael Dane (GSFC-720.0)[BUSINESS INTEGRA,
INC.]" wrote:
Our NDC domain is LDAP backed. Default is not.
Our keystone policy.json file is empty {}
Mike Moore, M.S.S.E.
Systems Engineer, Goddard Private Cloud
michael.d.mo...@nasa.gov
Hydrogen fusion brightens my day.
On 10/18/18, 7:24 PM, "Chris Apsey" wrote:
We are using multiple keystone domains - still can't reproduce this.
Do you happen to have a customized keystone policy.json?
Worst case, I would launch a devstack of your targeted release. If you
can't reproduce the issue there, you would at least know its caused by
a
nonstandard config rather than a bug (or at least not a bug that's
present
when using a default config)
On October 18, 2018 18:50:12 iain MacDonnell
wrote:
> That all looks fine.
>
> I believe that the "default" policy applies in place of any that's not
> explicitly specified - i.e. "if there's no matching policy below, you
> need to have the admin role to be able to do it". I do have that line
in
> my policy.json, and I cannot reproduce your problem (see below).
>
> I'm not using domains (other than "default"). I wonder if that's a
factor...
>
> ~iain
>
>
> $ openstack user create --password foo user1
> +-+--+
> | Field | Value|
> +-+--+
> | domain_id | default |
> | enabled | True |
> | id | d18c0031ec56430499a2d690cb1f125c |
> | name| user1|
> | options | {} |
> | password_expires_at | None |
> +-+--+
> $ openstack user create --password foo user2
> +-+--+
> | Field | Value|
> +-+--+
> | domain_id | default |
> | enabled | True |
> | id | be9f1061a5104abd834eabe98dff055d |
> | name| user2|
> | options | {} |
> | password_expires_at | None |
> +-+--+
> $ openstack project create project1
> +-+--+
> | Field | Value|
> +-+--+
> | description | |
> | domain_id | default |
> | enabled | True |
> | id | 826876d6d3724018bae6253c7f540cb3
Re: [Openstack-operators] Glance Image Visibility Issue? - Non admin users can see private images from other tenants
Our NDC domain is LDAP backed. Default is not.
Our keystone policy.json file is empty {}
Mike Moore, M.S.S.E.
Systems Engineer, Goddard Private Cloud
michael.d.mo...@nasa.gov
Hydrogen fusion brightens my day.
On 10/18/18, 7:24 PM, "Chris Apsey" wrote:
We are using multiple keystone domains - still can't reproduce this.
Do you happen to have a customized keystone policy.json?
Worst case, I would launch a devstack of your targeted release. If you
can't reproduce the issue there, you would at least know its caused by a
nonstandard config rather than a bug (or at least not a bug that's present
when using a default config)
On October 18, 2018 18:50:12 iain MacDonnell
wrote:
> That all looks fine.
>
> I believe that the "default" policy applies in place of any that's not
> explicitly specified - i.e. "if there's no matching policy below, you
> need to have the admin role to be able to do it". I do have that line in
> my policy.json, and I cannot reproduce your problem (see below).
>
> I'm not using domains (other than "default"). I wonder if that's a
factor...
>
> ~iain
>
>
> $ openstack user create --password foo user1
> +-+--+
> | Field | Value|
> +-+--+
> | domain_id | default |
> | enabled | True |
> | id | d18c0031ec56430499a2d690cb1f125c |
> | name| user1|
> | options | {} |
> | password_expires_at | None |
> +-+--+
> $ openstack user create --password foo user2
> +-+--+
> | Field | Value|
> +-+--+
> | domain_id | default |
> | enabled | True |
> | id | be9f1061a5104abd834eabe98dff055d |
> | name| user2|
> | options | {} |
> | password_expires_at | None |
> +-+--+
> $ openstack project create project1
> +-+--+
> | Field | Value|
> +-+--+
> | description | |
> | domain_id | default |
> | enabled | True |
> | id | 826876d6d3724018bae6253c7f540cb3 |
> | is_domain | False|
> | name| project1 |
> | parent_id | default |
> | tags| [] |
> +-+--+
> $ openstack project create project2
> +-+--+
> | Field | Value|
> +-+--+
> | description | |
> | domain_id | default |
> | enabled | True |
> | id | b446b93ac6e24d538c1943acbdd13cb2 |
> | is_domain | False|
> | name| project2 |
> | parent_id | default |
> | tags| [] |
> +-+--+
> $ openstack role add --user user1 --project project1 _member_
> $ openstack role add --user user2 --project project2 _member_
> $ export OS_PASSWORD=foo
> $ export OS_USERNAME=user1
> $ export OS_PROJECT_NAME=project1
> $ openstack image list
> +--+++
> | ID | Name | Status |
> +--+++
> | ad497523-b497-4500-8e6c-b5fb12a30cee | cirros | active |
> +--+++
> $ openstack image create --private image1
>
+--+--+
> | Field| Value
> |
>
+--+--+
> |
Re: [Openstack-operators] [Openstack-sigs] [all] Naming the T release of OpenStack
On Thu, Oct 18, 2018 at 05:35:39PM +1100, Tony Breeds wrote: > Hello all, > As per [1] the nomination period for names for the T release have > now closed (actually 3 days ago sorry). The nominated names and any > qualifying remarks can be seen at2]. > > Proposed Names > * Tarryall > * Teakettle > * Teller > * Telluride > * Thomas > * Thornton > * Tiger > * Tincup > * Timnath > * Timber > * Tiny Town > * Torreys > * Trail > * Trinidad > * Treasure > * Troublesome > * Trussville > * Turret > * Tyrone > > Proposed Names that do not meet the criteria > * Train I have re-worked my openstack/governance change[1] to ask the TC to accept adding Train to the poll as (partially) described in [2]. I present the names above to the community and Foundation marketing team for consideration. The list above does contain Train, clearly if the TC do not approve [1] Train will not be included in the poll when created. I apologise for any offence or slight caused by my previous email in this thread. It was well intentioned albeit, with hindsight, poorly thought through. Yours Tony. [1] https://review.openstack.org/#/c/611511/ [2] https://governance.openstack.org/tc/reference/release-naming.html#release-name-criteria signature.asc Description: PGP signature ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
[Openstack-operators] Fleio - OpenStack billing - ver. 1.1 released
Hello, We've just released Fleio version 1.1. Fleio is a billing solution and control panel for OpenStack public clouds and traditional web hosters. Fleio software automates the entire process for cloud users. New customers can use Fleio to sign up for an account, pay invoices, add credit to their account, as well as create and manage cloud resources such as virtual machines, storage and networking. Full feature list: https://fleio.com#features You can see an online demo: https://fleio.com/demo And sign-up for a free trial: https://fleio.com/signup Cheers! - Adrian Andreias https://fleio.com ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
[Openstack-operators] [heat][cinder] How to create stack snapshot including volumes
Hi List, I'd like to take snapshots of heat stacks including the volumes. >From what I found until now, this should be possible. You just have to configure some parts of OpenStack. I enabled cinder-backup with ceph backend. Backups from volumes are working. I configured heat to include the option backups_enabled = True. When I use openstack stack snapshot create, I get a snapshot but no backups of my volumes. I don't get any error messages in heat. Debug logging didn't help either. OpenStack version is Pike on Ubuntu installed with openstack-ansible. heat version is 9.0.3. So this should also include this bugfix: https://bugs.launchpad.net/heat/+bug/1687006 Is anybody using this feature? What am I missing? Best regards Christian ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
[Openstack-operators] osops-tools-monitoring Dependency problems
Hi! I'm a long time user of monitoring-for-openstack, also known as oschecks. Concretely, I used a version from 2015 with OpenStack python client libraries from Kilo. Now I have upgraded them to Mitaka and it got broken. Even the latest oschecks don't work. I didn't quite expect that, given that there are several commits from this year e.g. by Nagasai Vinaykumar Kapalavai and paramite. Can one of them or some other user step up and say what version of OpenStack clients is oschecks working with? Ideally, write it down in requirements.txt so that it will be reproducible? Also, some documentation of what is the minimal set of parameters would also come in handy. Thanks a lot, Tomas from Homeatcloud The error messages are as absurd as: oschecks-check_glance_api --os_auth_url='http://10.1.101.30:5000/v2.0' --os_username=monitoring --os_password=XXX --os_tenant_name=monitoring CRITICAL: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/oschecks/utils.py", line 121, in safe_run method() File "/usr/lib/python2.7/dist-packages/oschecks/glance.py", line 29, in _check_glance_api glance = utils.Glance() File "/usr/lib/python2.7/dist-packages/oschecks/utils.py", line 177, in __init__ self.glance.parser = self.glance.get_base_parser(sys.argv) TypeError: get_base_parser() takes exactly 1 argument (2 given) (I can see 4 parameters on the command line.) ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
