Hi Matt, Thanks for your kindly reply. I mispelled Newton in last email.
I did not meet any problems with RabbitMQ, and the nova instances works well. From the error log I pasted, it seems something goes wrong when novaclient is trying to communicate with nova api. On Wed, Jan 18, 2017 at 09:22:04PM -0700, Matt Fischer wrote: > Trove works fine with neutron. I would look deeper into your logs. Do you > have any errors about issues with Rabbit message timeouts? If so your guest > may have issues talking to Rabbit. That seems to be a common issue. > > On Wed, Jan 18, 2017 at 8:59 PM, Amrith Kumar <amrith.ku...@gmail.com> > wrote: > > > Sorry Wang Sen, why do you say Trove is not ready for Neutron"? It has > > worked with Neutron for some releases now. > > > > This does not appear to be at all related to Neutron. > > > > -amrith > > > > -- > > amrith.ku...@gmail.com > > On Jan 18, 2017 10:56 PM, "Wang Sen" <wang...@linux.vnet.ibm.com> wrote: > > > >> Hi all, > >> > >> I met the resource not found error when I'm creating a database > >> instance. The instance stays on build status and turns to error status > >> after timeout. > >> > >> I know trove is not ready for neuton. Is there a work around for this > >> issue ? Thanks in advance. > >> > >> Below is the detailed information. > >> > >> Error Log > >> ========= > >> > >> /var/log/trove/trove-taskmanager.log: > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task [-] Error > >> during Manager.publish_exists_event > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task Traceback > >> (most recent call last): > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File > >> "/usr/lib/python2.7/dist-packages/oslo_service/periodic_task.py", line > >> 220, in run_periodic_tasks > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task > >> task(self, context) > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File > >> "/usr/lib/python2.7/dist-packages/trove/taskmanager/manager.py", line > >> 429, in publish_exists_event > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task > >> self.admin_context) > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File > >> "/usr/lib/python2.7/dist-packages/trove/extensions/mgmt/instances/models.py", > >> line 178, in publish_exist_events > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task > >> notifications = transformer() > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File > >> "/usr/lib/python2.7/dist-packages/trove/extensions/mgmt/instances/models.py", > >> line 271, in __call__ > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task > >> client=self.nova_client) > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File > >> "/usr/lib/python2.7/dist-packages/trove/extensions/mgmt/instances/models.py", > >> line 40, in load_mgmt_instances > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task > >> mgmt_servers = client.servers.list(search_opts={'all_tenants': 1}) > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File > >> "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 835, > >> in list > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task > >> "servers") > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File > >> "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 249, in _list > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task resp, > >> body = self.api.client.get(url) > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File > >> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 480, in get > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task return > >> self._cs_request(url, 'GET', **kwargs) > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File > >> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 436, in > >> _cs_request > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task > >> self.authenticate() > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File > >> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 619, in > >> authenticate > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task > >> self._v2_auth(auth_url) > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File > >> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 684, in > >> _v2_auth > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task return > >> self._authenticate(url, body) > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File > >> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 697, in > >> _authenticate > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task > >> **kwargs) > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File > >> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 431, in > >> _time_request > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task resp, > >> body = self.request(url, method, **kwargs) > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File > >> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 425, in > >> request > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task raise > >> exceptions.from_response(resp, body, url, method) > >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task NotFound: > >> The resource could not be found. (HTTP 404) > >> > >> Openstack Cluster > >> ================= > >> > >> openstack version: Neuton > >> trove version: 2.5.0 > >> $ root@kvm-215:~# trove --version > >> 2.5.0 > >> $ root@kvm-215:~# openstack --version > >> openstack 3.2.0 > >> > >> Controller Node: ubuntu 16.04, 9.181.129.215 > >> Compute Node: ubuntu 16.04, 9.181.129.213 > >> > >> Manage network: 192.168.1.0/24 > >> Provider network: 9.181.129.0/24 > >> > >> endpoints: > >> > >> $ openstack endpoint list > >> > >> +----------------------------------+-----------+------------ > >> --+--------------+---------+-----------+-------------------- > >> --------------------------+ > >> | ID | Region | Service Name | Service > >> Type | Enabled | Interface | URL | > >> +----------------------------------+-----------+------------ > >> --+--------------+---------+-----------+-------------------- > >> --------------------------+ > >> | 08fa008d727643428800f10b31e32e3b | RegionOne | glance | image > >> | True | admin | http://controller:9292 > >> | > >> | 26cd2828179e4ca58b4cda80127889c2 | RegionOne | cinder | volume > >> | True | admin | http://controller:8776/v1/%(tenant_id)s > >> | > >> | 3148ed89e7e241b287d5dc68960d8052 | RegionOne | cinderv2 | > >> volumev2 | True | admin | http://controller:8776/v2/%(te > >> nant_id)s | > >> | 42d5aac6c0a349c5a167280459b16549 | RegionOne | trove | > >> database | True | public | http://controller:8779/v1.0/%( > >> tenant_id)s | > >> | 4600804165954819944a8c0ff9fe5f7d | RegionOne | trove | > >> database | True | admin | http://controller:8779/v1.0/%( > >> tenant_id)s | > >> | 4b7e6ba4db9d406d9f008a5c71700d94 | RegionOne | nova | compute > >> | True | internal | http://controller:8774/v2.1/%(tenant_id)s > >> | > >> | 4c35a14f4c8a4224aede30f91af25647 | RegionOne | glance | image > >> | True | public | http://controller:9292 > >> | > >> | 5a2ea322cadb4a4881db689015dfa182 | RegionOne | neutron | network > >> | True | public | http://controller:9696 > >> | > >> | 688688d146ee4d0eb93da67d108dd6a8 | RegionOne | neutron | network > >> | True | internal | http://controller:9696 > >> | > >> | 6db0991bd4e9439a87ed08e696dfbf35 | RegionOne | cinderv2 | > >> volumev2 | True | internal | http://controller:8776/v2/%(te > >> nant_id)s | > >> | 721c045a5acb44c8a67545bd880e05aa | RegionOne | cinder | volume > >> | True | internal | http://controller:8776/v1/%(tenant_id)s > >> | > >> | 74261ec81ed642b680b35a101902f9c2 | RegionOne | swift | > >> object-store | True | public | http://controller:8080/v1/AUTH > >> _%(tenant_id)s | > >> | 74f5a84c8bb344bda8f3dce5570001d4 | RegionOne | neutron | network > >> | True | admin | http://controller:9696 > >> | > >> | 85a295d376584809ac2c8adde8c59afa | RegionOne | cinder | volume > >> | True | public | http://controller:8776/v1/%(tenant_id)s > >> | > >> | 886c00a92bbf4b6d9244427d4f7a0e8d | RegionOne | swift | > >> object-store | True | admin | http://controller:8080/v1 > >> | > >> | 8ccce1c3736a4c0493ffee8f07f016b0 | RegionOne | keystone | > >> identity | True | admin | http://controller:35357/v3/ > >> | > >> | b59590340879457cb6311d3044387635 | RegionOne | swift | > >> object-store | True | internal | http://controller:8080/v1/AUTH > >> _%(tenant_id)s | > >> | b80a4a3dda8344ac992644d2c8dc167b | RegionOne | trove | > >> database | True | internal | http://controller:8779/v1.0/%( > >> tenant_id)s | > >> | bcfc25d9ec6d4cbfb10746cbd3caf153 | RegionOne | keystone | > >> identity | True | internal | http://controller:35357/v3/ > >> | > >> | bf565acdf1b64ea8a079a7e49f37331a | RegionOne | keystone | > >> identity | True | public | http://controller:5000/v3/ > >> | > >> | dbcb99659b9f428aae909f77cb22bd1c | RegionOne | cinderv2 | > >> volumev2 | True | public | http://controller:8776/v2/%(te > >> nant_id)s | > >> | de93f0d04ffa43b0bdc93da396044a06 | RegionOne | nova | compute > >> | True | admin | http://controller:8774/v2.1/%(tenant_id)s > >> | > >> | f9e8afbd6fe747619fd325982930e782 | RegionOne | glance | image > >> | True | internal | http://controller:9292 > >> | > >> | fb13a0896e724ac4b972531a3cc46853 | RegionOne | nova | compute > >> | True | public | http://controller:8774/v2.1/%(tenant_id)s > >> | > >> +----------------------------------+-----------+------------ > >> --+--------------+---------+-----------+-------------------- > >> --------------------------+ > >> > >> Trove Configuration > >> =================== > >> > >> trove.conf: > >> [DEFAULT] > >> debug = False > >> auth_strategy = keystone > >> add_addresses = True > >> bind_host = 0.0.0.0 > >> bind_port = 8779 > >> rpc_backend=rabbit > >> control_exchange = trove > >> db_api_implementation = "trove.db.sqlalchemy.api" > >> trove_auth_url = http://controller:5000/v3 > >> nova_compute_url = http://controller:8774/v2.1 > >> cinder_url = http://controller:8776/v1 > >> swift_url = http://controller:8080/v1/AUTH_ > >> notifier_queue_hostname = controller > >> os_region_name = RegionOne > >> network_label_regex = .* > >> trove_volume_support = True > >> block_device_mapping = vdb > >> device_path = /dev/vdb > >> max_accepted_volume_size = 10 > >> max_instances_per_tenant = 5 > >> max_volumes_per_tenant = 100 > >> max_backups_per_tenant = 5 > >> volume_time_out=30 > >> http_get_rate = 200 > >> http_post_rate = 200 > >> http_put_rate = 200 > >> http_delete_rate = 200 > >> http_mgmt_post_rate = 200 > >> trove_dns_support = False > >> dns_account_id = 123456 > >> dns_auth_url = http://127.0.0.1:5000/v2.0 > >> dns_username = user > >> dns_passkey = password > >> dns_ttl = 3600 > >> dns_domain_name = 'trove.com.' > >> dns_domain_id = 11111111-1111-1111-1111-111111111111 > >> dns_driver = trove.dns.designate.driver.DesignateDriver > >> dns_instance_entry_factory = trove.dns.designate.driver.Des > >> ignateInstanceEntryFactory > >> dns_endpoint_url = http://127.0.0.1/v1/ > >> dns_service_type = dns > >> network_driver = trove.network.nova.NovaNetwork > >> default_neutron_networks = > >> taskmanager_queue = taskmanager > >> admin_roles = admin > >> agent_heartbeat_time = 10 > >> agent_call_low_timeout = 5 > >> agent_call_high_timeout = 150 > >> reboot_time_out = 60 > >> api_paste_config = /etc/trove/api-paste.ini > >> log_dir = /var/log/trove > >> [database] > >> connection = mysql+pymysql://trove:pass4trove@controller/trove > >> idle_timeout = 3600 > >> [profiler] > >> [ssl] > >> [t_host=controller > >> rabbit_userid=openstack > >> rabbit_password=RABBITPASS > >> [mysql] > >> root_on_create = False > >> tcp_ports = 3306 > >> volume_support = True > >> device_path = /dev/vdb > >> ignore_users = os_admin, root > >> ignore_dbs = mysql, information_schema, performance_schema > >> [redis] > >> tcp_ports = 6379 > >> volume_support = False > >> [cassandra] > >> tcp_ports = 7000, 7001, 9042, 9160 > >> volume_support = True > >> device_path = /dev/vdb > >> [couchbase] > >> tcp_ports = 8091, 8092, 4369, 11209-11211, 21100-21199 > >> volume_support = True > >> device_path = /dev/vdb > >> [mongodb] > >> tcp_ports = 2500, 27017, 27019 > >> volume_support = True > >> device_path = /dev/vdb > >> num_config_servers_per_cluster = 1 > >> num_query_routers_per_cluster = 1 > >> [vertica] > >> tcp_ports = 5433, 5434, 22, 5444, 5450, 4803 > >> udp_ports = 5433, 4803, 4804, 6453 > >> volume_support = True > >> device_path = /dev/vdb > >> cluster_support = True > >> cluster_member_count = 3 > >> api_strategy = trove.common.strategies.cluster.experimental.vertica.api. > >> VerticaAPIStrategy > >> [cors] > >> [cors.subdomain] > >> [keystone_authtoken] > >> auth_uri = http://controller:5000 > >> auth_url = http://controller:35357 > >> auth_type = password > >> project_domain_name = default > >> user_domain_name = default > >> project_name = service > >> username = trove > >> password = pass4trove > >> oslo_messaging_rabbit] > >> > >> > >> trove-taskmanager.conf: > >> [DEFAULT] > >> debug = False > >> update_status_on_fail = True > >> rpc_backend=rabbit > >> control_exchange = trove > >> db_api_implementation = trove.db.sqlalchemy.api > >> trove_auth_url = http://controller:5000/v3 > >> nova_compute_url = http://controller:8774/v2.1 > >> cinder_url = http://controller:8776/v1 > >> swift_url = http://controller:8080/v1/AUTH_ > >> notifier_queue_hostname = controller > >> os_region_name = RegionOne > >> trove_volume_support = True > >> block_device_mapping = vdb > >> device_path = /dev/vdb > >> mount_point = /var/lib/mysql > >> volume_time_out=30 > >> server_delete_time_out=480 > >> use_nova_server_config_drive = True > >> nova_proxy_admin_user = admin > >> nova_proxy_admin_pass = frobisher86 > >> nova_proxy_admin_tenant_name = service > >> taskmanager_manager=trove.taskmanager.manager.Manager > >> exists_notification_transformer = trove.extensions.mgmt.instance > >> s.models.NovaNotificationTransformer > >> exists_notification_ticks = 30 > >> notification_service_id = mysql:2f3ff068-2bfb-4f70-9a9d-a6bb65bc084b > >> trove_dns_support = False > >> dns_account_id = 123456 > >> dns_auth_url = http://controller:5000/v2.0 > >> dns_username = user > >> dns_passkey = password > >> dns_ttl = 3600 > >> dns_domain_name = 'trove.com.' > >> dns_domain_id = 11111111-1111-1111-1111-111111111111 > >> dns_driver = trove.dns.designate.driver.DesignateDriver > >> dns_instance_entry_factory = trove.dns.designate.driver.Des > >> ignateInstanceEntryFactory > >> dns_endpoint_url = http://controller/v1/ > >> dns_service_type = dns > >> network_driver=trove.network.neutron.NeutronDriver > >> default_neutron_networks = > >> trove_security_groups_support = True > >> trove_security_group_rule_cidr = 0.0.0.0/0 > >> agent_heartbeat_time = 10 > >> agent_call_low_timeout = 5 > >> agent_call_high_timeout = 150 > >> agent_replication_snapshot_timeout = 36000 > >> use_nova_server_volume = False > >> network_label_regex = .* > >> template_path = /etc/trove/templates/ > >> pydev_debug = disabled > >> [database] > >> connection = mysql+pymysql://trove:pass4trove@controller/trove > >> idle_timeout = 3600 > >> [profiler] > >> [oslo_messaging_rabbit] > >> rabbit_host=controller > >> rabbit_userid=openstack > >> rabbit_password=pass4rabbit > >> [mysql] > >> icmp = True > >> tcp_ports = 3306 > >> volume_support = True > >> device_path = /dev/vdb > >> [redis] > >> tcp_ports = 6379 > >> volume_support = False > >> [cassandra] > >> tcp_ports = 7000, 7001, 9042, 9160 > >> volume_support = True > >> device_path = /dev/vdb > >> [couchbase] > >> tcp_ports = 8091, 8092, 4369, 11209-11211, 21100-21199 > >> volume_support = True > >> device_path = /dev/vdb > >> [mongodb] > >> volume_support = True > >> device_path = /dev/vdb > >> [vertica] > >> tcp_ports = 5433, 5434, 22, 5444, 5450, 4803 > >> udp_ports = 5433, 4803, 4804, 6453 > >> volume_support = True > >> device_path = /dev/vdb > >> mount_point = /var/lib/vertica > >> taskmanager_strategy = trove.common.strategies.cluste > >> r.experimental.vertica.taskmanager.VerticaTaskManagerStrategy > >> > >> > >> trove-conductor.conf: > >> [DEFAULT] > >> debug = False > >> trove_auth_url = http://controller:5000/v3 > >> notifier_queue_hostname = controller > >> log_dir = /var/log/trove > >> conductor_manager = trove.conductor.manager.Manager > >> rpc_backend=rabbit > >> control_exchange = trove > >> [profiler] > >> [database] > >> connection = mysql+pymysql://trove:pass4trove@controller/trove > >> [oslo_messaging_rabbit] > >> rabbit_host=controller > >> rabbit_userid=openstack > >> rabbit_password=RABBITPASS > >> > >> > >> trove-guestagent.conf: > >> [DEFAULT] > >> debug = True > >> log_file = trove-guestagent.log > >> log_dir = /var/log/trove > >> ignore_users = os_admin > >> control_exchange = trove > >> > >> rabbit_host = controller > >> rabbit_userid = openstack > >> rabbit_password = RABBITPASS > >> nova_proxy_admin_user = admin > >> nova_proxy_admin_pass = ADMIN_PASS > >> nova_proxy_admin_tenant_name = service > >> trove_auth_url = http://controller:35357/v3 > >> > >> > >> ____________________________________________________________ > >> ______________ > >> OpenStack Development Mailing List (not for usage questions) > >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib > >> e > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev